Info

Permalink
Function
smp_call_function_many
First occurrence
2021-01-02
Last occurrence
2026-02-06
State
NEW
Components
kernel
Quality
-51

Statistics

Operating system Count
CentOS 7 13
Architecture Count
x86_64 13
Related packages Count
kernel 13
 0:3.10.0-1160.108.1.el7 3
 0:3.10.0-1160.6.1.el7 2
 0:3.10.0-1127.19.1.el7 1
 0:3.10.0-693.el7 1
 0:3.10.0-862.11.6.el7 1
 0:3.10.0-1160.119.1.el7 1
 0:3.10.0-1160.62.1.el7 1
 0:3.10.0-862.el7 1
 0:3.10.0-1160.53.1.el7 1
 0:3.10.0-1160.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1396960
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffabf1e23a
Build id: 3.10.0-1160.108.1.el7.x86_64
0x20a
2
? drain_pages
vmlinux 0xffffffffabfd28c0
Build id: 3.10.0-1160.108.1.el7.x86_64
0xb0
3
on_each_cpu_mask
vmlinux 0xffffffffabf1e389
Build id: 3.10.0-1160.108.1.el7.x86_64
0x29
4
drain_all_pages
vmlinux 0xffffffffabfcf6c5
Build id: 3.10.0-1160.108.1.el7.x86_64
0xb5
5
__alloc_pages_nodemask
vmlinux 0xffffffffabfd46ab
Build id: 3.10.0-1160.108.1.el7.x86_64
0x8db
6
alloc_pages_current
vmlinux 0xffffffffac025a48
Build id: 3.10.0-1160.108.1.el7.x86_64
0x98
7
bio_copy_user_iov
vmlinux 0xffffffffac09c5a7
Build id: 3.10.0-1160.108.1.el7.x86_64
0x1c7
8
bio_copy_kern
vmlinux 0xffffffffac09c81c
Build id: 3.10.0-1160.108.1.el7.x86_64
0x4c
9
blk_rq_map_kern
vmlinux 0xffffffffac170b3d
Build id: 3.10.0-1160.108.1.el7.x86_64
0x7d
10
scsi_execute
vmlinux 0xffffffffac30955f
Build id: 3.10.0-1160.108.1.el7.x86_64
0x13f
11
scsi_execute_req_flags
vmlinux 0xffffffffac30b4fe
Build id: 3.10.0-1160.108.1.el7.x86_64
0x8e
12
sr_check_events
sr_mod 0xffffffffc03e935c
Build id: 3.10.0-1160.108.1.el7.x86_64
0xbc
13
cdrom_check_events
cdrom 0xffffffffc04e006e
Build id: 3.10.0-1160.108.1.el7.x86_64
0x1e
14
sr_block_check_events
sr_mod 0xffffffffc03ea101
Build id: 3.10.0-1160.108.1.el7.x86_64
0xb1
15
disk_check_events
vmlinux 0xffffffffac17f256
Build id: 3.10.0-1160.108.1.el7.x86_64
0x66
16
disk_events_workfn
vmlinux 0xffffffffac17f3a6
Build id: 3.10.0-1160.108.1.el7.x86_64
0x16
17
process_one_work
vmlinux 0xffffffffabec32ef
Build id: 3.10.0-1160.108.1.el7.x86_64
0x17f
18
worker_thread
vmlinux 0xffffffffabec4436
Build id: 3.10.0-1160.108.1.el7.x86_64
0x126
19
? manage_workers.isra.26
vmlinux 0xffffffffabec4310
Build id: 3.10.0-1160.108.1.el7.x86_64
0x2b0
20
kthread
vmlinux 0xffffffffabecb621
Build id: 3.10.0-1160.108.1.el7.x86_64
0xd1
21
? insert_kthread_work
vmlinux 0xffffffffabecb550
Build id: 3.10.0-1160.108.1.el7.x86_64
0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffac5c51dd
Build id: 3.10.0-1160.108.1.el7.x86_64
0x7
23
? insert_kthread_work
vmlinux 0xffffffffabecb550
Build id: 3.10.0-1160.108.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [kworker/4:3:5299]
Modules linked in: i2dtrack(OE) iscsi_target_mod target_core_mod sfs(OE) nbd(E) dattobd(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper sg ablk_helper cryptd ppdev vmw_balloon joydev pcspkr parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm drm nfit ata_piix mptspi libnvdimm scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih crc32c_intel libata mptbase e1000 serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 5299 Comm: kworker/4:3 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.108.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: events_freezable disk_events_workfn
task: ffff92d974a1c200 ti: ffff92d957184000 task.ti: ffff92d957184000
RIP: 0010:[<ffffffffabf1e23a>]  [<ffffffffabf1e23a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff92d957187948  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff92da79620678
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff92d957187980 R08: ffff92da76663c00 R09: ffffffffac19c7b9
R10: ffff92da7971f160 R11: ffffec1502e26c00 R12: 0000000000000296
R13: 0000000000000018 R14: ffff92d9571878b8 R15: 0000000000000296
FS:  0000000000000000(0000) GS:ffff92da79700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055975ee9f31a CR3: 0000000131ef4000 CR4: 0000000000760fe0
PKRU: 00000000
Call Trace:
[<ffffffffabfd28c0>] ? drain_pages+0xb0/0xb0
[<ffffffffabf1e389>] on_each_cpu_mask+0x29/0x70
[<ffffffffabfcf6c5>] drain_all_pages+0xb5/0xd0
[<ffffffffabfd46ab>] __alloc_pages_nodemask+0x8db/0xbf0
[<ffffffffac025a48>] alloc_pages_current+0x98/0x110
[<ffffffffac09c5a7>] bio_copy_user_iov+0x1c7/0x390
[<ffffffffac09c81c>] bio_copy_kern+0x4c/0xf0
[<ffffffffac170b3d>] blk_rq_map_kern+0x7d/0x190
[<ffffffffac30955f>] scsi_execute+0x13f/0x170
[<ffffffffac30b4fe>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc03e935c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc04e006e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc03ea101>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffffac17f256>] disk_check_events+0x66/0x1a0
[<ffffffffac17f3a6>] disk_events_workfn+0x16/0x20
[<ffffffffabec32ef>] process_one_work+0x17f/0x440
[<ffffffffabec4436>] worker_thread+0x126/0x3c0
[<ffffffffabec4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffffabecb621>] kthread+0xd1/0xe0
[<ffffffffabecb550>] ? insert_kthread_work+0x40/0x40
[<ffffffffac5c51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffabecb550>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 8e 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 80 19 b5 ac f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d 18 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #1204373
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8c716c0a
Build id: 3.10.0-1160.el7.x86_64
0x20a
2
? drain_pages
vmlinux 0xffffffff8c7c74a0
Build id: 3.10.0-1160.el7.x86_64
0xb0
3
on_each_cpu_mask
vmlinux 0xffffffff8c716d59
Build id: 3.10.0-1160.el7.x86_64
0x29
4
drain_all_pages
vmlinux 0xffffffff8c7c4385
Build id: 3.10.0-1160.el7.x86_64
0xb5
5
__alloc_pages_slowpath
vmlinux 0xffffffff8cd7c6b2
Build id: 3.10.0-1160.el7.x86_64
0x510
6
__alloc_pages_nodemask
vmlinux 0xffffffff8c7c8d76
Build id: 3.10.0-1160.el7.x86_64
0x436
7
alloc_pages_current
vmlinux 0xffffffff8c8189d8
Build id: 3.10.0-1160.el7.x86_64
0x98
8
bio_copy_user_iov
vmlinux 0xffffffff8c88cac7
Build id: 3.10.0-1160.el7.x86_64
0x1c7
9
bio_copy_kern
vmlinux 0xffffffff8c88cd2c
Build id: 3.10.0-1160.el7.x86_64
0x4c
10
blk_rq_map_kern
vmlinux 0xffffffff8c95b42d
Build id: 3.10.0-1160.el7.x86_64
0x7d
11
scsi_execute
vmlinux 0xffffffff8cae9a67
Build id: 3.10.0-1160.el7.x86_64
0x137
12
scsi_execute_req_flags
vmlinux 0xffffffff8caeb8de
Build id: 3.10.0-1160.el7.x86_64
0x8e
13
sr_check_events
sr_mod 0xffffffffc027231c
Build id: 3.10.0-1160.el7.x86_64
0xbc
14
cdrom_check_events
cdrom 0xffffffffc01e905e
Build id: 3.10.0-1160.el7.x86_64
0x1e
15
sr_block_check_events
sr_mod 0xffffffffc0273091
Build id: 3.10.0-1160.el7.x86_64
0xb1
16
disk_check_events
vmlinux 0xffffffff8c969496
Build id: 3.10.0-1160.el7.x86_64
0x66
17
disk_events_workfn
vmlinux 0xffffffff8c9695d6
Build id: 3.10.0-1160.el7.x86_64
0x16
18
process_one_work
vmlinux 0xffffffff8c6bdc4f
Build id: 3.10.0-1160.el7.x86_64
0x17f
19
worker_thread
vmlinux 0xffffffff8c6bed66
Build id: 3.10.0-1160.el7.x86_64
0x126
20
? manage_workers.isra.26
vmlinux 0xffffffff8c6bec40
Build id: 3.10.0-1160.el7.x86_64
0x2a0
21
kthread
vmlinux 0xffffffff8c6c5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
22
? insert_kthread_work
vmlinux 0xffffffff8c6c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
23
ret_from_fork_nospec_begin
vmlinux 0xffffffff8cd93df7
Build id: 3.10.0-1160.el7.x86_64
0x21
24
? insert_kthread_work
vmlinux 0xffffffff8c6c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/3:2:61879]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi crc32_pclmul snd_ac97_codec ghash_clmulni_intel aesni_intel ac97_bus lrw gf128mul snd_seq glue_helper snd_seq_device snd_pcm ablk_helper cryptd
snd_timer snd soundcore ppdev sg pcspkr joydev vmw_balloon parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix nfit crct10dif_pclmul drm crct10dif_common libata crc32c_intel e1000 libnvdimm mptspi scsi_transport_spi serio_raw mptscsih mptbase drm_panel_orientation_quirks fuse
CPU: 3 PID: 61879 Comm: kworker/3:2 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: events_freezable disk_events_workfn
task: ffff8ecabbdda100 ti: ffff8ec972308000 task.ti: ffff8ec972308000
RIP: 0010:[<ffffffff8c716c0a>]  [<ffffffff8c716c0a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8ec97230b8e0  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000800008c7c74a0 RCX: ffff8ecb5d620650
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8ec97230b918 R08: ffff8ec4b2f5c000 R09: ffffffff8c985f99
R10: ffff8ecb5d6df160 R11: ffffe8151fbd2200 R12: 0000000000000286
R13: 0000000000000286 R14: ffff8ec97230b890 R15: 0000000001320122
FS:  0000000000000000(0000) GS:ffff8ecb5d6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000009b08a70 CR3: 0000000799e28000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8c7c74a0>] ? drain_pages+0xb0/0xb0
[<ffffffff8c716d59>] on_each_cpu_mask+0x29/0x70
[<ffffffff8c7c4385>] drain_all_pages+0xb5/0xc0
[<ffffffff8cd7c6b2>] __alloc_pages_slowpath+0x510/0x729
[<ffffffff8c7c8d76>] __alloc_pages_nodemask+0x436/0x450
[<ffffffff8c8189d8>] alloc_pages_current+0x98/0x110
[<ffffffff8c88cac7>] bio_copy_user_iov+0x1c7/0x390
[<ffffffff8c88cd2c>] bio_copy_kern+0x4c/0xe0
[<ffffffff8c95b42d>] blk_rq_map_kern+0x7d/0x180
[<ffffffff8cae9a67>] scsi_execute+0x137/0x170
[<ffffffff8caeb8de>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc027231c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc01e905e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc0273091>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffff8c969496>] disk_check_events+0x66/0x190
[<ffffffff8c9695d6>] disk_events_workfn+0x16/0x20
[<ffffffff8c6bdc4f>] process_one_work+0x17f/0x440
[<ffffffff8c6bed66>] worker_thread+0x126/0x3c0
[<ffffffff8c6bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8c6c5c21>] kthread+0xd1/0xe0
[<ffffffff8c6c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff8cd93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8c6c5b50>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 1e 89 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 35 8d f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 88 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #982055
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffbc7170da
Build id: 3.10.0-1160.53.1.el7.x86_64
0x20a
2
? drain_pages
vmlinux 0xffffffffbc7c78e0
Build id: 3.10.0-1160.53.1.el7.x86_64
0xb0
3
on_each_cpu_mask
vmlinux 0xffffffffbc717229
Build id: 3.10.0-1160.53.1.el7.x86_64
0x29
4
drain_all_pages
vmlinux 0xffffffffbc7c47c5
Build id: 3.10.0-1160.53.1.el7.x86_64
0xb5
5
__alloc_pages_nodemask
vmlinux 0xffffffffbc7c964b
Build id: 3.10.0-1160.53.1.el7.x86_64
0x8cb
6
alloc_pages_current
vmlinux 0xffffffffbc8193b8
Build id: 3.10.0-1160.53.1.el7.x86_64
0x98
7
bio_copy_user_iov
vmlinux 0xffffffffbc88d457
Build id: 3.10.0-1160.53.1.el7.x86_64
0x1c7
8
bio_copy_kern
vmlinux 0xffffffffbc88d6bc
Build id: 3.10.0-1160.53.1.el7.x86_64
0x4c
9
blk_rq_map_kern
vmlinux 0xffffffffbc95bf2d
Build id: 3.10.0-1160.53.1.el7.x86_64
0x7d
10
scsi_execute
vmlinux 0xffffffffbcaea947
Build id: 3.10.0-1160.53.1.el7.x86_64
0x137
11
scsi_execute_req_flags
vmlinux 0xffffffffbcaec7be
Build id: 3.10.0-1160.53.1.el7.x86_64
0x8e
12
sr_check_events
sr_mod 0xffffffffc049c31c
Build id: 3.10.0-1160.53.1.el7.x86_64
0xbc
13
cdrom_check_events
cdrom 0xffffffffc05ae05e
Build id: 3.10.0-1160.53.1.el7.x86_64
0x1e
14
sr_block_check_events
sr_mod 0xffffffffc049d091
Build id: 3.10.0-1160.53.1.el7.x86_64
0xb1
15
disk_check_events
vmlinux 0xffffffffbc96a056
Build id: 3.10.0-1160.53.1.el7.x86_64
0x66
16
disk_events_workfn
vmlinux 0xffffffffbc96a196
Build id: 3.10.0-1160.53.1.el7.x86_64
0x16
17
process_one_work
vmlinux 0xffffffffbc6bde8f
Build id: 3.10.0-1160.53.1.el7.x86_64
0x17f
18
worker_thread
vmlinux 0xffffffffbc6befa6
Build id: 3.10.0-1160.53.1.el7.x86_64
0x126
19
? manage_workers.isra.26
vmlinux 0xffffffffbc6bee80
Build id: 3.10.0-1160.53.1.el7.x86_64
0x2a0
20
kthread
vmlinux 0xffffffffbc6c5e61
Build id: 3.10.0-1160.53.1.el7.x86_64
0xd1
21
? insert_kthread_work
vmlinux 0xffffffffbc6c5d90
Build id: 3.10.0-1160.53.1.el7.x86_64
0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffbcd95ddd
Build id: 3.10.0-1160.53.1.el7.x86_64
0x7
23
? insert_kthread_work
vmlinux 0xffffffffbc6c5d90
Build id: 3.10.0-1160.53.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [bareos-sd:6127]
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 24s! [kworker/5:0:15111]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter tun bridge stp llc nfsv3 nfs_acl nfs lockd grace fscache vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod ahci crc_t10dif crct10dif_generic ata_piix libahci
drm nfit crct10dif_pclmul crct10dif_common libnvdimm libata crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks fuse
CPU: 5 PID: 15111 Comm: kworker/5:0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: events_freezable disk_events_workfn
task: ffff9298c0281080 ti: ffff9297c471c000 task.ti: ffff9297c471c000
RIP: 0010:[<ffffffffbc7170da>]  [<ffffffffbc7170da>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9297c471f948  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 00000000000000fc RCX: ffff929a7fca06a0
RDX: 0000000000000002 RSI: 0000000000000006 RDI: 0000000000000000
RBP: ffff9297c471f980 R08: ffff929939376000 R09: ffffffffbc986bf9
R10: ffff929a7fd5f160 R11: ffffc9c004ccb400 R12: 0000000000000296
R13: ffff9297c471f8f8 R14: 0000000001320122 R15: 000000004c785c03
FS:  0000000000000000(0000) GS:ffff929a7fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3cc51d49d0 CR3: 0000000293c3c000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffbc7c78e0>] ? drain_pages+0xb0/0xb0
[<ffffffffbc717229>] on_each_cpu_mask+0x29/0x70
[<ffffffffbc7c47c5>] drain_all_pages+0xb5/0xc0
[<ffffffffbc7c964b>] __alloc_pages_nodemask+0x8cb/0xbe0
[<ffffffffbc8193b8>] alloc_pages_current+0x98/0x110
[<ffffffffbc88d457>] bio_copy_user_iov+0x1c7/0x390
[<ffffffffbc88d6bc>] bio_copy_kern+0x4c/0xe0
[<ffffffffbc95bf2d>] blk_rq_map_kern+0x7d/0x180
[<ffffffffbcaea947>] scsi_execute+0x137/0x170
[<ffffffffbcaec7be>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc049c31c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc05ae05e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc049d091>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffffbc96a056>] disk_check_events+0x66/0x190
[<ffffffffbc96a196>] disk_events_workfn+0x16/0x20
[<ffffffffbc6bde8f>] process_one_work+0x17f/0x440
[<ffffffffbc6befa6>] worker_thread+0x126/0x3c0
[<ffffffffbc6bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffbc6c5e61>] kthread+0xd1/0xe0
[<ffffffffbc6c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffbcd95ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffbc6c5d90>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 ce 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 35 bd f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #873940
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa8d1722a
Build id: 3.10.0-1160.62.1.el7.x86_64
0x20a
2
? drain_pages
vmlinux 0xffffffffa8dc78f0
Build id: 3.10.0-1160.62.1.el7.x86_64
0xb0
3
on_each_cpu_mask
vmlinux 0xffffffffa8d17379
Build id: 3.10.0-1160.62.1.el7.x86_64
0x29
4
drain_all_pages
vmlinux 0xffffffffa8dc47d5
Build id: 3.10.0-1160.62.1.el7.x86_64
0xb5
5
__alloc_pages_nodemask
vmlinux 0xffffffffa8dc965b
Build id: 3.10.0-1160.62.1.el7.x86_64
0x8cb
6
alloc_pages_current
vmlinux 0xffffffffa8e193a8
Build id: 3.10.0-1160.62.1.el7.x86_64
0x98
7
bio_copy_user_iov
vmlinux 0xffffffffa8e8d2e7
Build id: 3.10.0-1160.62.1.el7.x86_64
0x1c7
8
bio_copy_kern
vmlinux 0xffffffffa8e8d54c
Build id: 3.10.0-1160.62.1.el7.x86_64
0x4c
9
blk_rq_map_kern
vmlinux 0xffffffffa8f5bd0d
Build id: 3.10.0-1160.62.1.el7.x86_64
0x7d
10
scsi_execute
vmlinux 0xffffffffa90ea727
Build id: 3.10.0-1160.62.1.el7.x86_64
0x137
11
scsi_execute_req_flags
vmlinux 0xffffffffa90ec59e
Build id: 3.10.0-1160.62.1.el7.x86_64
0x8e
12
sr_check_events
sr_mod 0xffffffffc028731c
Build id: 3.10.0-1160.62.1.el7.x86_64
0xbc
13
cdrom_check_events
cdrom 0xffffffffc027b05e
Build id: 3.10.0-1160.62.1.el7.x86_64
0x1e
14
sr_block_check_events
sr_mod 0xffffffffc0288091
Build id: 3.10.0-1160.62.1.el7.x86_64
0xb1
15
disk_check_events
vmlinux 0xffffffffa8f69e36
Build id: 3.10.0-1160.62.1.el7.x86_64
0x66
16
disk_events_workfn
vmlinux 0xffffffffa8f69f76
Build id: 3.10.0-1160.62.1.el7.x86_64
0x16
17
process_one_work
vmlinux 0xffffffffa8cbdfdf
Build id: 3.10.0-1160.62.1.el7.x86_64
0x17f
18
worker_thread
vmlinux 0xffffffffa8cbf0f6
Build id: 3.10.0-1160.62.1.el7.x86_64
0x126
19
? manage_workers.isra.26
vmlinux 0xffffffffa8cbefd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x2a0
20
kthread
vmlinux 0xffffffffa8cc5fb1
Build id: 3.10.0-1160.62.1.el7.x86_64
0xd1
21
? system_call_after_swapgs
vmlinux 0xffffffffa9399ec9
Build id: 3.10.0-1160.62.1.el7.x86_64
0x96
22
? insert_kthread_work
vmlinux 0xffffffffa8cc5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x40
23
ret_from_fork_nospec_begin
vmlinux 0xffffffffa9399df7
Build id: 3.10.0-1160.62.1.el7.x86_64
0x21
24
? insert_kthread_work
vmlinux 0xffffffffa8cc5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:3:3374]
Modules linked in: veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) snd_intel8x0 snd_ac97_codec crc32_pclmul ghash_clmulni_intel ac97_bus aesni_intel snd_seq lrw snd_seq_device gf128mul snd_pcm glue_helper ppdev ablk_helper snd_timer sg cryptd snd soundcore parport_pc i2c_piix4 parport joydev pcspkr video ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel e1000 serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3374 Comm: kworker/0:3 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Workqueue: events_freezable disk_events_workfn
task: ffff8c8c19382100 ti: ffff8c8b7b5b0000 task.ti: ffff8c8b7b5b0000
RIP: 0010:[<ffffffffa8d1722a>]  [<ffffffffa8d1722a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8c8b7b5b3948  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 00000000000000fc RCX: ffff8c8d5fd1fc10
RDX: 0000000000000002 RSI: 0000000000000003 RDI: 0000000000000000
RBP: ffff8c8b7b5b3980 R08: ffff8c8cbfd4d400 R09: ffffffffa8f869d9
R10: ffff8c8d5fc1f160 R11: ffffd84343398400 R12: 0000000000000296
R13: ffff8c8b7b5b38f8 R14: ffffffffa8d33605 R15: 000000005e5e29e6
FS:  0000000000000000(0000) GS:ffff8c8d5fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000eede2000 CR3: 00000000d1386000 CR4: 00000000000606f0
Call Trace:
[<ffffffffa8dc78f0>] ? drain_pages+0xb0/0xb0
[<ffffffffa8d17379>] on_each_cpu_mask+0x29/0x70
[<ffffffffa8dc47d5>] drain_all_pages+0xb5/0xc0
[<ffffffffa8dc965b>] __alloc_pages_nodemask+0x8cb/0xbe0
[<ffffffffa8e193a8>] alloc_pages_current+0x98/0x110
[<ffffffffa8e8d2e7>] bio_copy_user_iov+0x1c7/0x390
[<ffffffffa8e8d54c>] bio_copy_kern+0x4c/0xe0
[<ffffffffa8f5bd0d>] blk_rq_map_kern+0x7d/0x180
[<ffffffffa90ea727>] scsi_execute+0x137/0x170
[<ffffffffa90ec59e>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc028731c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc027b05e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc0288091>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffffa8f69e36>] disk_check_events+0x66/0x190
[<ffffffffa8f69f76>] disk_events_workfn+0x16/0x20
[<ffffffffa8cbdfdf>] process_one_work+0x17f/0x440
[<ffffffffa8cbf0f6>] worker_thread+0x126/0x3c0
[<ffffffffa8cbefd0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa8cc5fb1>] kthread+0xd1/0xe0
[<ffffffffa9399ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffa8cc5ee0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa9399df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa8cc5ee0>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 7e 87 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 18 95 a9 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 4d 87 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #408456
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa77165b2
Build id: 3.10.0-1160.6.1.el7.x86_64
0x102
2
? drain_pages
vmlinux 0xffffffffa77c74e0
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb0
3
? drain_pages
vmlinux 0xffffffffa77c74e0
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb0
4
? drain_pages
vmlinux 0xffffffffa77c74e0
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb0
5
smp_call_function_single
vmlinux 0xffffffffa77166cf
Build id: 3.10.0-1160.6.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa7985e25
Build id: 3.10.0-1160.6.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa7716c7b
Build id: 3.10.0-1160.6.1.el7.x86_64
0x22b
8
? drain_pages
vmlinux 0xffffffffa77c74e0
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb0
9
on_each_cpu_mask
vmlinux 0xffffffffa7716da9
Build id: 3.10.0-1160.6.1.el7.x86_64
0x29
10
drain_all_pages
vmlinux 0xffffffffa77c43c5
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb5
11
__alloc_pages_slowpath
vmlinux 0xffffffffa7d7c772
Build id: 3.10.0-1160.6.1.el7.x86_64
0x510
12
__alloc_pages_nodemask
vmlinux 0xffffffffa77c8db6
Build id: 3.10.0-1160.6.1.el7.x86_64
0x436
13
alloc_pages_current
vmlinux 0xffffffffa7818a18
Build id: 3.10.0-1160.6.1.el7.x86_64
0x98
14
bio_copy_user_iov
vmlinux 0xffffffffa788cb07
Build id: 3.10.0-1160.6.1.el7.x86_64
0x1c7
15
bio_copy_kern
vmlinux 0xffffffffa788cd6c
Build id: 3.10.0-1160.6.1.el7.x86_64
0x4c
16
blk_rq_map_kern
vmlinux 0xffffffffa795b46d
Build id: 3.10.0-1160.6.1.el7.x86_64
0x7d
17
scsi_execute
vmlinux 0xffffffffa7ae9ab7
Build id: 3.10.0-1160.6.1.el7.x86_64
0x137
18
scsi_execute_req_flags
vmlinux 0xffffffffa7aeb92e
Build id: 3.10.0-1160.6.1.el7.x86_64
0x8e
19
sr_check_events
sr_mod 0xffffffffc05fa31c
Build id: 3.10.0-1160.6.1.el7.x86_64
0xbc
20
cdrom_check_events
cdrom 0xffffffffc05e905e
Build id: 3.10.0-1160.6.1.el7.x86_64
0x1e
21
sr_block_check_events
sr_mod 0xffffffffc05fb091
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb1
22
disk_check_events
vmlinux 0xffffffffa79694d6
Build id: 3.10.0-1160.6.1.el7.x86_64
0x66
23
disk_events_workfn
vmlinux 0xffffffffa7969616
Build id: 3.10.0-1160.6.1.el7.x86_64
0x16
24
process_one_work
vmlinux 0xffffffffa76bdc4f
Build id: 3.10.0-1160.6.1.el7.x86_64
0x17f
25
worker_thread
vmlinux 0xffffffffa76bed66
Build id: 3.10.0-1160.6.1.el7.x86_64
0x126
26
? manage_workers.isra.26
vmlinux 0xffffffffa76bec40
Build id: 3.10.0-1160.6.1.el7.x86_64
0x2a0
27
kthread
vmlinux 0xffffffffa76c5c21
Build id: 3.10.0-1160.6.1.el7.x86_64
0xd1
28
? insert_kthread_work
vmlinux 0xffffffffa76c5b50
Build id: 3.10.0-1160.6.1.el7.x86_64
0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffffa7d94df7
Build id: 3.10.0-1160.6.1.el7.x86_64
0x21
30
? insert_kthread_work
vmlinux 0xffffffffa76c5b50
Build id: 3.10.0-1160.6.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/1:1:21645]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi ac97_bus snd_seq ppdev crc32_pclmul snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel sg lrw gf128mul glue_helper
ablk_helper cryptd snd_timer snd soundcore pcspkr joydev vmw_balloon i2c_piix4 vmw_vmci parport_pc parport binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix serio_raw libata e1000 mptspi scsi_transport_spi mptscsih mptbase nfit libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 21645 Comm: kworker/1:1 Kdump: loaded Not tainted 3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: events_freezable disk_events_workfn
task: ffff949be2383180 ti: ffff949cb2bc0000 task.ti: ffff949cb2bc0000
RIP: 0010:[<ffffffffa77165b2>]  [<ffffffffa77165b2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff949cb2bc3850  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff949cb2bc3820 RCX: 0000000000000001
RDX: ffffffffa7e137d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff949cb2bc38a0 R08: ffffffffa7e137c8 R09: 0000000000000000
R10: 0000000000001e89 R11: fff00000f0000000 R12: ffff949d5d9823b0
R13: 0000000000000000 R14: 0000000000000032 R15: 0000000000000400
FS:  0000000000000000(0000) GS:ffff949d75640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f51e2be4200 CR3: 00000001e2346000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa77c74e0>] ? drain_pages+0xb0/0xb0
[<ffffffffa77c74e0>] ? drain_pages+0xb0/0xb0
[<ffffffffa77c74e0>] ? drain_pages+0xb0/0xb0
[<ffffffffa77166cf>] smp_call_function_single+0x5f/0xa0
[<ffffffffa7985e25>] ? cpumask_next_and+0x35/0x50
[<ffffffffa7716c7b>] smp_call_function_many+0x22b/0x270
[<ffffffffa77c74e0>] ? drain_pages+0xb0/0xb0
[<ffffffffa7716da9>] on_each_cpu_mask+0x29/0x70
[<ffffffffa77c43c5>] drain_all_pages+0xb5/0xc0
[<ffffffffa7d7c772>] __alloc_pages_slowpath+0x510/0x729
[<ffffffffa77c8db6>] __alloc_pages_nodemask+0x436/0x450
[<ffffffffa7818a18>] alloc_pages_current+0x98/0x110
[<ffffffffa788cb07>] bio_copy_user_iov+0x1c7/0x390
[<ffffffffa788cd6c>] bio_copy_kern+0x4c/0xe0
[<ffffffffa795b46d>] blk_rq_map_kern+0x7d/0x180
[<ffffffffa7ae9ab7>] scsi_execute+0x137/0x170
[<ffffffffa7aeb92e>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc05fa31c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc05e905e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc05fb091>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffffa79694d6>] disk_check_events+0x66/0x190
[<ffffffffa7969616>] disk_events_workfn+0x16/0x20
[<ffffffffa76bdc4f>] process_one_work+0x17f/0x440
[<ffffffffa76bed66>] worker_thread+0x126/0x3c0
[<ffffffffa76bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa76c5c21>] kthread+0xd1/0xe0
[<ffffffffa76c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffffa7d94df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa76c5b50>] ? insert_kthread_work+0x40/0x40
Code: 89 de 48 03 14 c5 e0 13 35 a8 48 89 df e8 e7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #331929
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffafb16c5a
Build id: 3.10.0-1160.6.1.el7.x86_64
0x20a
2
? drain_pages
vmlinux 0xffffffffafbc74e0
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb0
3
on_each_cpu_mask
vmlinux 0xffffffffafb16da9
Build id: 3.10.0-1160.6.1.el7.x86_64
0x29
4
drain_all_pages
vmlinux 0xffffffffafbc43c5
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb5
5
__alloc_pages_slowpath
vmlinux 0xffffffffb017c772
Build id: 3.10.0-1160.6.1.el7.x86_64
0x510
6
__alloc_pages_nodemask
vmlinux 0xffffffffafbc8db6
Build id: 3.10.0-1160.6.1.el7.x86_64
0x436
7
alloc_pages_current
vmlinux 0xffffffffafc18a18
Build id: 3.10.0-1160.6.1.el7.x86_64
0x98
8
bio_copy_user_iov
vmlinux 0xffffffffafc8cb07
Build id: 3.10.0-1160.6.1.el7.x86_64
0x1c7
9
bio_copy_kern
vmlinux 0xffffffffafc8cd6c
Build id: 3.10.0-1160.6.1.el7.x86_64
0x4c
10
blk_rq_map_kern
vmlinux 0xffffffffafd5b46d
Build id: 3.10.0-1160.6.1.el7.x86_64
0x7d
11
scsi_execute
vmlinux 0xffffffffafee9ab7
Build id: 3.10.0-1160.6.1.el7.x86_64
0x137
12
scsi_execute_req_flags
vmlinux 0xffffffffafeeb92e
Build id: 3.10.0-1160.6.1.el7.x86_64
0x8e
13
sr_check_events
sr_mod 0xffffffffc052631c
Build id: 3.10.0-1160.6.1.el7.x86_64
0xbc
14
cdrom_check_events
cdrom 0xffffffffc04ba05e
Build id: 3.10.0-1160.6.1.el7.x86_64
0x1e
15
sr_block_check_events
sr_mod 0xffffffffc0527091
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb1
16
disk_check_events
vmlinux 0xffffffffafd694d6
Build id: 3.10.0-1160.6.1.el7.x86_64
0x66
17
disk_events_workfn
vmlinux 0xffffffffafd69616
Build id: 3.10.0-1160.6.1.el7.x86_64
0x16
18
process_one_work
vmlinux 0xffffffffafabdc4f
Build id: 3.10.0-1160.6.1.el7.x86_64
0x17f
19
worker_thread
vmlinux 0xffffffffafabed66
Build id: 3.10.0-1160.6.1.el7.x86_64
0x126
20
? manage_workers.isra.26
vmlinux 0xffffffffafabec40
Build id: 3.10.0-1160.6.1.el7.x86_64
0x2a0
21
kthread
vmlinux 0xffffffffafac5c21
Build id: 3.10.0-1160.6.1.el7.x86_64
0xd1
22
? insert_kthread_work
vmlinux 0xffffffffafac5b50
Build id: 3.10.0-1160.6.1.el7.x86_64
0x40
23
ret_from_fork_nospec_begin
vmlinux 0xffffffffb0194df7
Build id: 3.10.0-1160.6.1.el7.x86_64
0x21
24
? insert_kthread_work
vmlinux 0xffffffffafac5b50
Build id: 3.10.0-1160.6.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [kworker/1:1:2265]
Modules linked in: sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper joydev pcspkr cryptd ppdev sg vmw_balloon parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul e1000 crct10dif_common ata_piix crc32c_intel serio_raw libata mptspi scsi_transport_spi mptscsih mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 2265 Comm: kworker/1:1 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
Workqueue: events_freezable disk_events_workfn
task: ffff893f45aed280 ti: ffff893e9604c000 task.ti: ffff893e9604c000
RIP: 0010:[<ffffffffafb16c5a>]  [<ffffffffafb16c5a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff893e9604f8e0  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000c0000bff8a1c0 RCX: ffff893fbfc20600
RDX: 0000000000000000 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff893e9604f918 R08: ffff893efdb41000 R09: ffffffffafd85fd9
R10: ffff893fbfc5f160 R11: ffffd7434548c000 R12: 0000000000000286
R13: 0000000000000286 R14: ffff893e9604f890 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff893fbfc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8461fe7a48 CR3: 00000001c9864000 CR4: 00000000000607e0
Call Trace:
[<ffffffffafbc74e0>] ? drain_pages+0xb0/0xb0
[<ffffffffafb16da9>] on_each_cpu_mask+0x29/0x70
[<ffffffffafbc43c5>] drain_all_pages+0xb5/0xc0
[<ffffffffb017c772>] __alloc_pages_slowpath+0x510/0x729
[<ffffffffafbc8db6>] __alloc_pages_nodemask+0x436/0x450
[<ffffffffafc18a18>] alloc_pages_current+0x98/0x110
[<ffffffffafc8cb07>] bio_copy_user_iov+0x1c7/0x390
[<ffffffffafc8cd6c>] bio_copy_kern+0x4c/0xe0
[<ffffffffafd5b46d>] blk_rq_map_kern+0x7d/0x180
[<ffffffffafee9ab7>] scsi_execute+0x137/0x170
[<ffffffffafeeb92e>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc052631c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc04ba05e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc0527091>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffffafd694d6>] disk_check_events+0x66/0x190
[<ffffffffafd69616>] disk_events_workfn+0x16/0x20
[<ffffffffafabdc4f>] process_one_work+0x17f/0x440
[<ffffffffafabed66>] worker_thread+0x126/0x3c0
[<ffffffffafabec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffafac5c21>] kthread+0xd1/0xe0
[<ffffffffafac5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffffb0194df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffafac5b50>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 ce 88 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 75 b0 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 88 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #187820
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810f99a2
Build id: 3.10.0-693.el7.x86_64
0x202
2
? drain_pages
vmlinux 0xffffffff8118b530
Build id: 3.10.0-693.el7.x86_64
0xb0
3
on_each_cpu_mask
vmlinux 0xffffffff810f9ae9
Build id: 3.10.0-693.el7.x86_64
0x29
4
drain_all_pages
vmlinux 0xffffffff81188475
Build id: 3.10.0-693.el7.x86_64
0xb5
5
__alloc_pages_slowpath
vmlinux 0xffffffff8169fbc4
Build id: 3.10.0-693.el7.x86_64
0x510
6
__alloc_pages_nodemask
vmlinux 0xffffffff8118cd85
Build id: 3.10.0-693.el7.x86_64
0x405
7
alloc_pages_current
vmlinux 0xffffffff811d1108
Build id: 3.10.0-693.el7.x86_64
0x98
8
bio_copy_user_iov
vmlinux 0xffffffff8123bdb7
Build id: 3.10.0-693.el7.x86_64
0x1c7
9
? get_request
vmlinux 0xffffffff812f7fa8
Build id: 3.10.0-693.el7.x86_64
0x398
10
bio_copy_kern
vmlinux 0xffffffff8123c01c
Build id: 3.10.0-693.el7.x86_64
0x4c
11
blk_rq_map_kern
vmlinux 0xffffffff812fe76d
Build id: 3.10.0-693.el7.x86_64
0x7d
12
scsi_execute
vmlinux 0xffffffff8146b2f7
Build id: 3.10.0-693.el7.x86_64
0x137
13
scsi_execute_req_flags
vmlinux 0xffffffff8146ca0e
Build id: 3.10.0-693.el7.x86_64
0x8e
14
sr_check_events
sr_mod 0xffffffffc020431c
Build id: 3.10.0-693.el7.x86_64
0xbc
15
cdrom_check_events
cdrom 0xffffffffc012e058
Build id: 3.10.0-693.el7.x86_64
0x18
16
sr_block_check_events
sr_mod 0xffffffffc02047c9
Build id: 3.10.0-693.el7.x86_64
0x39
17
disk_check_events
vmlinux 0xffffffff81309db0
Build id: 3.10.0-693.el7.x86_64
0x60
18
disk_events_workfn
vmlinux 0xffffffff81309ee6
Build id: 3.10.0-693.el7.x86_64
0x16
19
process_one_work
vmlinux 0xffffffff810a881a
Build id: 3.10.0-693.el7.x86_64
0x17a
20
worker_thread
vmlinux 0xffffffff810a94e6
Build id: 3.10.0-693.el7.x86_64
0x126
21
? manage_workers.isra.24
vmlinux 0xffffffff810a93c0
Build id: 3.10.0-693.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64
0xcf
23
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
0x40
24
ret_from_fork
vmlinux 0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64
0x58
25
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:2:24326]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 coretemp snd_rawmidi snd_ac97_codec ac97_bus snd_seq crc32_pclmul snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel lrw gf128mul snd_timer glue_helper ablk_helper snd cryptd soundcore pcspkr sg joydev ppdev vmw_balloon parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel mptspi scsi_transport_spi libata mptscsih serio_raw e1000 mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 24326 Comm: kworker/0:2 Tainted: G             L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2012
Workqueue: events_freezable disk_events_workfn
task: ffff88020c69bf40 ti: ffff880056be4000 task.ti: ffff880056be4000
RIP: 0010:[<ffffffff810f99a2>]  [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff880056be78e0  EFLAGS: 00000202
RAX: 0000000000000003 RBX: 0000000000000282 RCX: ffff8802366da0e0
RDX: 0000000000000003 RSI: 0000000000000040 RDI: 0000000000000000
RBP: ffff880056be7918 R08: ffff88017fc55800 R09: ffffffff813227d9
R10: ffff880236619c80 R11: ffffea0006108000 R12: 0000000000000282
R13: ffff880056be7890 R14: 0000000000000078 R15: 0000000001320122
FS:  0000000000000000(0000) GS:ffff880236600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007ff4de51c980 CR3: 00000000019f2000 CR4: 00000000000007f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000100000040 ffffffff81f404c0 ffffffff81f404c0 ffffffff8118b530
0000000000000000 0000000000000000 ffff88023ffd7000 ffff880056be7948
ffffffff810f9ae9 0000000000000007 0000000000000040 0000000000000040
Call Trace:
[<ffffffff8118b530>] ? drain_pages+0xb0/0xb0
[<ffffffff810f9ae9>] on_each_cpu_mask+0x29/0x60
[<ffffffff81188475>] drain_all_pages+0xb5/0xc0
[<ffffffff8169fbc4>] __alloc_pages_slowpath+0x510/0x724
[<ffffffff8118cd85>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811d1108>] alloc_pages_current+0x98/0x110
[<ffffffff8123bdb7>] bio_copy_user_iov+0x1c7/0x390
[<ffffffff812f7fa8>] ? get_request+0x398/0x760
[<ffffffff8123c01c>] bio_copy_kern+0x4c/0xe0
[<ffffffff812fe76d>] blk_rq_map_kern+0x7d/0x180
[<ffffffff8146b2f7>] scsi_execute+0x137/0x170
[<ffffffff8146ca0e>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc020431c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc012e058>] cdrom_check_events+0x18/0x30 [cdrom]
[<ffffffffc02047c9>] sr_block_check_events+0x39/0x50 [sr_mod]
[<ffffffff81309db0>] disk_check_events+0x60/0x180
[<ffffffff81309ee6>] disk_events_workfn+0x16/0x20
[<ffffffff810a881a>] process_one_work+0x17a/0x440
[<ffffffff810a94e6>] worker_thread+0x126/0x3c0
[<ffffffff810a93c0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 26 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89 
Complete report #24037
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff86516f72
Build id: 3.10.0-1127.19.1.el7.x86_64
0x102
2
? drain_pages
vmlinux 0xffffffff865c7900
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
3
? drain_pages
vmlinux 0xffffffff865c7900
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
4
? drain_pages
vmlinux 0xffffffff865c7900
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
smp_call_function_single
vmlinux 0xffffffff8651708f
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff86785005
Build id: 3.10.0-1127.19.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff8651763b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x22b
8
? drain_pages
vmlinux 0xffffffff865c7900
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
9
on_each_cpu_mask
vmlinux 0xffffffff86517769
Build id: 3.10.0-1127.19.1.el7.x86_64
0x29
10
drain_all_pages
vmlinux 0xffffffff865c4775
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb5
11
__alloc_pages_slowpath
vmlinux 0xffffffff86b7b315
Build id: 3.10.0-1127.19.1.el7.x86_64
0x510
12
__alloc_pages_nodemask
vmlinux 0xffffffff865c91f6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x436
13
alloc_pages_current
vmlinux 0xffffffff86618ea8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x98
14
bio_copy_user_iov
vmlinux 0xffffffff8668bfe7
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1c7
15
bio_copy_kern
vmlinux 0xffffffff8668c24c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4c
16
blk_rq_map_kern
vmlinux 0xffffffff8675a6fd
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7d
17
scsi_execute
vmlinux 0xffffffff868e8a77
Build id: 3.10.0-1127.19.1.el7.x86_64
0x137
18
scsi_execute_req_flags
vmlinux 0xffffffff868ea8ee
Build id: 3.10.0-1127.19.1.el7.x86_64
0x8e
19
sr_check_events
sr_mod 0xffffffffc039631c
Build id: 3.10.0-1127.19.1.el7.x86_64
0xbc
20
cdrom_check_events
cdrom 0xffffffffc033c05e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1e
21
sr_block_check_events
sr_mod 0xffffffffc0397091
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb1
22
disk_check_events
vmlinux 0xffffffff867686b6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x66
23
disk_events_workfn
vmlinux 0xffffffff867687f6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x16
24
process_one_work
vmlinux 0xffffffff864be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
25
worker_thread
vmlinux 0xffffffff864bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
26
? manage_workers.isra.26
vmlinux 0xffffffff864bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
27
kthread
vmlinux 0xffffffff864c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
28
? insert_kthread_work
vmlinux 0xffffffff864c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffff86b92d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
30
? insert_kthread_work
vmlinux 0xffffffff864c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/1:1:5227]
Modules linked in: tcp_diag inet_diag d2dctl(POE) fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter talpa_vfshook(OE) talpa_pedconnector(OE) talpa_pedevice(OE) ip6t_REJECT nf_reject_ipv6 talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) ipt_REJECT nf_reject_ipv4 talpa_syscallhook(OE) xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack vmw_vsock_vmci_transport vsock ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul
vmw_balloon glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea sysfillrect sysimgblt fb_sys_fops crc_t10dif ttm crct10dif_generic ahci drm ata_piix nfit libahci crct10dif_pclmul libnvdimm crct10dif_common crc32c_intel libata serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 5227 Comm: kworker/1:1 Kdump: loaded Tainted: P           OE  ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
Workqueue: events_freezable disk_events_workfn
task: ffff9c3d7743b150 ti: ffff9c3d74b98000 task.ti: ffff9c3d74b98000
RIP: 0010:[<ffffffff86516f72>]  [<ffffffff86516f72>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9c3d74b9b850  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff9c3d74b9b820 RCX: 000000000000003e
RDX: 0000000000000003 RSI: 0000000000000002 RDI: 0000000000000282
RBP: ffff9c3d74b9b8a0 R08: ffffffff86c136c0 R09: 0000000000000000
R10: 0000000000000f00 R11: 000000000000001e R12: ffff9c3cf7a3cbb0
R13: 0000000000000000 R14: 0000000000000034 R15: 0000000000000400
FS:  0000000000000000(0000) GS:ffff9c3d7fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f53adc10f1b CR3: 00000000ba4c6000 CR4: 00000000001607e0
Call Trace:
[<ffffffff865c7900>] ? drain_pages+0xb0/0xb0
[<ffffffff865c7900>] ? drain_pages+0xb0/0xb0
[<ffffffff865c7900>] ? drain_pages+0xb0/0xb0
[<ffffffff8651708f>] smp_call_function_single+0x5f/0xa0
[<ffffffff86785005>] ? cpumask_next_and+0x35/0x50
[<ffffffff8651763b>] smp_call_function_many+0x22b/0x270
[<ffffffff865c7900>] ? drain_pages+0xb0/0xb0
[<ffffffff86517769>] on_each_cpu_mask+0x29/0x70
[<ffffffff865c4775>] drain_all_pages+0xb5/0xc0
[<ffffffff86b7b315>] __alloc_pages_slowpath+0x510/0x729
[<ffffffff865c91f6>] __alloc_pages_nodemask+0x436/0x450
[<ffffffff86618ea8>] alloc_pages_current+0x98/0x110
[<ffffffff8668bfe7>] bio_copy_user_iov+0x1c7/0x390
[<ffffffff8668c24c>] bio_copy_kern+0x4c/0xe0
[<ffffffff8675a6fd>] blk_rq_map_kern+0x7d/0x180
[<ffffffff868e8a77>] scsi_execute+0x137/0x170
[<ffffffff868ea8ee>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc039631c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc033c05e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc0397091>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffff867686b6>] disk_check_events+0x66/0x190
[<ffffffff867687f6>] disk_events_workfn+0x16/0x20
[<ffffffff864be6bf>] process_one_work+0x17f/0x440
[<ffffffff864bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff864bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff864c6691>] kthread+0xd1/0xe0
[<ffffffff864c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff86b92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff864c65c0>] ? insert_kthread_work+0x40/0x40
Code: 89 de 48 03 14 c5 a0 10 15 87 48 89 df e8 07 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00