Info

Permalink
Function
_raw_spin_unlock_irqrestore
First occurrence
2020-09-29
Last occurrence
2024-03-31
State
NEW
Components
kernel
Quality
-21

Statistics

Operating system Count
CentOS 7 47
Architecture Count
x86_64 47
Related packages Count
kernel 47
 0:3.10.0-1160.el7 12
 0:3.10.0-1160.119.1.el7 5
 0:3.10.0-1160.76.1.el7 3
 0:3.10.0-1062.4.1.el7 3
 0:3.10.0-1160.88.1.el7 2
 0:3.10.0-1127.19.1.el7 2
 0:3.10.0-1160.81.1.el7 2
 0:3.10.0-1160.25.1.el7 2
 0:3.10.0-957.el7 2
 0:3.10.0-1062.18.1.el7 1
 0:3.10.0-1062.9.1.el7 1
 0:3.10.0-1160.45.1.el7 1
 0:3.10.0-957.12.2.el7 1
 0:3.10.0-1127.18.2.el7 1
 0:3.10.0-1160.11.1.el7 1
 0:3.10.0-1127.el7 1
 0:3.10.0-1062.el7 1
 0:3.10.0-1160.114.2.el7 1
 0:3.10.0-1160.80.1.el7 1
 0:3.10.0-862.el7 1
 0:3.10.0-957.5.1.el7 1
 0:3.10.0-693.11.1.el7 1
 0:3.10.0-693.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #1410147
Frame # Function Binary Source or offset Line
1
rcu_process_gp_end
vmlinux 0xffffffff90361e8f
Build id: 3.10.0-1160.119.1.el7.x86_64
0x4f
2
rcu_process_callbacks
vmlinux 0xffffffff903630e5
Build id: 3.10.0-1160.119.1.el7.x86_64
0x95
3
__do_softirq
vmlinux 0xffffffff902a9595
Build id: 3.10.0-1160.119.1.el7.x86_64
0xf5
4
run_ksoftirqd
vmlinux 0xffffffff902a9768
Build id: 3.10.0-1160.119.1.el7.x86_64
0x38
5
smpboot_thread_fn
vmlinux 0xffffffff902d43fc
Build id: 3.10.0-1160.119.1.el7.x86_64
0x14c
6
? lg_double_unlock
vmlinux 0xffffffff902d42b0
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
7
kthread
vmlinux 0xffffffff902cb621
Build id: 3.10.0-1160.119.1.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffff902cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff909c51dd
Build id: 3.10.0-1160.119.1.el7.x86_64
0x7
10
? insert_kthread_work
vmlinux 0xffffffff902cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ksoftirqd/0:6]
Modules linked in: tcp_lp veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi iosf_mbi crc32_pclmul snd_ac97_codec ac97_bus ghash_clmulni_intel snd_seq ppdev vmw_balloon aesni_intel snd_seq_device snd_pcm lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr snd_timer snd sg vmw_vmci soundcore i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt crc32c_intel fb_sys_fops ata_piix ttm drm nfit mptspi libnvdimm scsi_transport_spi serio_raw libata mptscsih e1000 mptbase drm_panel_orientation_quirks fuse
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff90a6763ad280 ti: ffff90a675ce0000 task.ti: ffff90a675ce0000
RIP: 0010:[<ffffffff90361e8f>]  [<ffffffff90361e8f>] rcu_process_gp_end+0x4f/0x90
RSP: 0018:ffff90a675ce3da0  EFLAGS: 00000202
RAX: fffffffffffffed8 RBX: 0000000000000001 RCX: ffffffff90f5e460
RDX: ffff90a92d600000 RSI: ffff90a92d616200 RDI: 0000000000000202
RBP: ffff90a675ce3da8 R08: 00000000000200e0 R09: ffffffff904d7bac
R10: ffff90a92d6200e0 R11: ffffcbdcc25aea00 R12: 000000001cbae71a
R13: ffff90a92d61b530 R14: 0000000000000246 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff90a92d600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe42c5db000 CR3: 000000013db78000 CR4: 00000000007607f0
PKRU: 00000000
Call Trace:
[<ffffffff903630e5>] rcu_process_callbacks+0x95/0x580
[<ffffffff902a9595>] __do_softirq+0xf5/0x290
[<ffffffff902a9768>] run_ksoftirqd+0x38/0x60
[<ffffffff902d43fc>] smpboot_thread_fn+0x14c/0x1b0
[<ffffffff902d42b0>] ? lg_double_unlock+0x40/0x40
[<ffffffff902cb621>] kthread+0xd1/0xe0
[<ffffffff902cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff909c51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff902cb550>] ? insert_kthread_work+0x40/0x40
Code: 4c 8b 6e 18 49 8b 45 10 48 39 06 74 0f 49 89 fe 4c 89 ef e8 44 88 65 00 85 c0 75 20 48 89 df 57 9d 0f 1f 44 00 00 5b 41 5c 41 5d <41> 5e 5d c3 cc cc cc cc 66 0f 1f 84 00 00 00 00 00 4c 89 e2 4c 
Complete report #1320642
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb9bba385
Build id: 3.10.0-1160.119.1.el7.x86_64
0x15
2
rcu_process_callbacks
vmlinux 0xffffffffb95633b3
Build id: 3.10.0-1160.119.1.el7.x86_64
0x363
3
? __switch_to
vmlinux 0xffffffffb942c5ee
Build id: 3.10.0-1160.119.1.el7.x86_64
0xce
4
__do_softirq
vmlinux 0xffffffffb94a9595
Build id: 3.10.0-1160.119.1.el7.x86_64
0xf5
5
run_ksoftirqd
vmlinux 0xffffffffb94a9768
Build id: 3.10.0-1160.119.1.el7.x86_64
0x38
6
smpboot_thread_fn
vmlinux 0xffffffffb94d43fc
Build id: 3.10.0-1160.119.1.el7.x86_64
0x14c
7
? lg_double_unlock
vmlinux 0xffffffffb94d42b0
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
8
kthread
vmlinux 0xffffffffb94cb621
Build id: 3.10.0-1160.119.1.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffffb94cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffb9bc51dd
Build id: 3.10.0-1160.119.1.el7.x86_64
0x7
11
? insert_kthread_work
vmlinux 0xffffffffb94cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ksoftirqd/0:6]
<EOI> 
Modules linked in:
nfnetlink_queue nfnetlink_log bluetooth rfkill tcp_diag inet_diag binfmt_misc tcp_lp nls_utf8 isofs xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables ipip tunnel4 ip_tunnel ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter sunrpc crc32_pclmul ghash_clmulni_intel snd_intel8x0 snd_ac97_codec ac97_bus snd_seq joydev snd_seq_device ppdev snd_pcm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vboxguest(OE) sg pcspkr snd_timer snd parport_pc parport video soundcore i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm ata_piix ahci libahci drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8f82d8775280 ti: ffff8f82d87a4000 task.ti: ffff8f82d87a4000
RIP: 0010:[<ffffffffb9bba385>]  [<ffffffffb9bba385>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8f82d87a7da8  EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffffffffb94eab04 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff8f82d87a7da8 R08: ffff8f82973cb230 R09: 00000000000d8100
R10: ffffffffba07a3c0 R11: 0000000000000293 R12: 0000000000000000
R13: 0000000100000000 R14: 00000003e114f300 R15: ffff8f82e1000480
FS:  0000000000000000(0000) GS:ffff8f82e1400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000003e10008 CR3: 000000015666e000 CR4: 00000000000206f0
Call Trace:
[<ffffffffb95633b3>] rcu_process_callbacks+0x363/0x580
[<ffffffffb942c5ee>] ? __switch_to+0xce/0x590
[<ffffffffb94a9595>] __do_softirq+0xf5/0x290
[<ffffffffb94a9768>] run_ksoftirqd+0x38/0x60
[<ffffffffb94d43fc>] smpboot_thread_fn+0x14c/0x1b0
[<ffffffffb94d42b0>] ? lg_double_unlock+0x40/0x40
[<ffffffffb94cb621>] kthread+0xd1/0xe0
[<ffffffffb94cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffffb9bc51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffb94cb550>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 cc cc cc cc 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 cc cc cc cc 0f 1f 44 00 00 55 48 89 e5 48 
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:37:crtc-0] flip_done timed out
Complete report #1267418
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9ff89785
Build id: 3.10.0-1160.el7.x86_64
0x15
2
rcu_process_callbacks
vmlinux 0xffffffff9f95a6ab
Build id: 3.10.0-1160.el7.x86_64
0x35b
3
__do_softirq
vmlinux 0xffffffff9f8a4b95
Build id: 3.10.0-1160.el7.x86_64
0xf5
4
run_ksoftirqd
vmlinux 0xffffffff9f8a4d58
Build id: 3.10.0-1160.el7.x86_64
0x38
5
smpboot_thread_fn
vmlinux 0xffffffff9f8ce674
Build id: 3.10.0-1160.el7.x86_64
0x144
6
? lg_double_unlock
vmlinux 0xffffffff9f8ce530
Build id: 3.10.0-1160.el7.x86_64
0x40
7
kthread
vmlinux 0xffffffff9f8c5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffff9f8c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff9ff93df7
Build id: 3.10.0-1160.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffff9f8c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ksoftirqd/0:6]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device ppdev vboxguest(OE) snd_pcm intel_powerclamp joydev snd_timer pcspkr i2c_piix4 snd soundcore sg parport_pc parport video sch_fq_codel ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic crct10dif_common
cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata e1000 crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9d5d3cb45280 ti: ffff9d5d3cb70000 task.ti: ffff9d5d3cb70000
RIP: 0010:[<ffffffff9ff89785>]  [<ffffffff9ff89785>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9d5d3cb73da8  EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff9d5d3fc1ad40 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff9d5d3cb73da8 R08: ffff9d5d37b94200 R09: 0000000000009cdf
R10: ffffffffa047a380 R11: 0000000000000293 R12: ffffffff9f8e111e
R13: ffff9d5d3cb73d30 R14: 000000cd18b778a8 R15: ffffffff9f8e43cc
FS:  0000000000000000(0000) GS:ffff9d5d3fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0396d3a420 CR3: 0000000074166000 CR4: 00000000000206f0
Call Trace:
[<ffffffff9f95a6ab>] rcu_process_callbacks+0x35b/0x570
[<ffffffff9f8a4b95>] __do_softirq+0xf5/0x280
[<ffffffff9f8a4d58>] run_ksoftirqd+0x38/0x50
[<ffffffff9f8ce674>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff9f8ce530>] ? lg_double_unlock+0x40/0x40
[<ffffffff9f8c5c21>] kthread+0xd1/0xe0
[<ffffffff9f8c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff9ff93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9f8c5b50>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #1252506
Frame # Function Binary Source or offset Line
1
rcu_process_callbacks
vmlinux 0xffffffff81139ddb
Build id: 3.10.0-693.el7.x86_64
0x20b
2
__do_softirq
vmlinux 0xffffffff81090b3f
Build id: 3.10.0-693.el7.x86_64
0xef
3
run_ksoftirqd
vmlinux 0xffffffff81090d08
Build id: 3.10.0-693.el7.x86_64
0x38
4
smpboot_thread_fn
vmlinux 0xffffffff810b909f
Build id: 3.10.0-693.el7.x86_64
0x12f
5
? lg_double_unlock
vmlinux 0xffffffff810b8f70
Build id: 3.10.0-693.el7.x86_64
0x40
6
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64
0xcf
7
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
0x40
8
ret_from_fork
vmlinux 0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64
0x58
9
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ksoftirqd/0:3]
Modules linked in: ext4 mbcache jbd2 ppp_synctty ppp_async crc_ccitt ppp_generic slhc nf_conntrack_ftp nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd ppdev e1000 soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel
cryptd pcspkr joydev vmw_balloon sg nfit libnvdimm vmwgfx ttm drm_kms_helper syscopyarea sysfillrect parport_pc sysimgblt fb_sys_fops parport drm i2c_piix4 i2c_core shpchp vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3 Comm: ksoftirqd/0 Not tainted 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff880137429fa0 ti: ffff8801374c0000 task.ti: ffff8801374c0000
RIP: 0010:[<ffffffff81139ddb>]  [<ffffffff81139ddb>] rcu_process_callbacks+0x20b/0x550
RSP: 0000:ffff8801374c3db8  EFLAGS: 00010282
RAX: ffff880139610288 RBX: 0100000031901a01 RCX: ffff880139610240
RDX: ffff8800ad43bfd0 RSI: ffff88005df87df0 RDI: 0000000000000000
RBP: ffff8801374c3e08 R08: ffff8801374c3d48 R09: ffff880139610240
R10: 0000000000000000 R11: 0000000000000400 R12: 0000753000000000
R13: ffff8801374c3de0 R14: ffffffff8146e778 R15: ffffffffc02e348f
FS:  0000000000000000(0000) GS:ffff880139600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f591a105000 CR3: 00000001343ae000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88005df87df0 0000000000000000 ffff880139610288 ffff880137429fa0
ffff8801374c3fd8 0000000000000009 0000000000000009 ffffffff81a27440
ffffffff819eb108 0000000000000001 ffff8801374c3e78 ffffffff81090b3f
Call Trace:
[<ffffffff81090b3f>] __do_softirq+0xef/0x280
[<ffffffff81090d08>] run_ksoftirqd+0x38/0x50
[<ffffffff810b909f>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f70>] ? lg_double_unlock+0x40/0x40
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 01 4d 39 ec 7f 1d 48 8b 45 d0 48 8b 80 38 c0 ff ff a8 08 75 3b 48 8b 45 c8 8b 80 a4 04 00 00 85 c0 75 2d 4d 85 f6 74 28 4c 89 f2 <48> 8b 42 08 4c 8b 32 48 3d ff 0f 00 00 41 0f 18 0e 77 ba 48 29 
Complete report #1074518
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9b98ea15
Build id: 3.10.0-1160.76.1.el7.x86_64
0x15
2
rcu_process_callbacks
vmlinux 0xffffffff9b35ac2b
Build id: 3.10.0-1160.76.1.el7.x86_64
0x35b
3
? __switch_to
vmlinux 0xffffffff9b22b59e
Build id: 3.10.0-1160.76.1.el7.x86_64
0xce
4
__do_softirq
vmlinux 0xffffffff9b2a4d85
Build id: 3.10.0-1160.76.1.el7.x86_64
0xf5
5
run_ksoftirqd
vmlinux 0xffffffff9b2a4f48
Build id: 3.10.0-1160.76.1.el7.x86_64
0x38
6
smpboot_thread_fn
vmlinux 0xffffffff9b2ce9e4
Build id: 3.10.0-1160.76.1.el7.x86_64
0x144
7
? lg_double_unlock
vmlinux 0xffffffff9b2ce8a0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
8
kthread
vmlinux 0xffffffff9b2c5f91
Build id: 3.10.0-1160.76.1.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffff9b2c5ec0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffff9b999ddd
Build id: 3.10.0-1160.76.1.el7.x86_64
0x7
11
? insert_kthread_work
vmlinux 0xffffffff9b2c5ec0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 10552s! [ksoftirqd/2:20]
Modules linked in: sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq crc32_pclmul snd_seq_device ghash_clmulni_intel ppdev snd_pcm aesni_intel lrw gf128mul glue_helper snd_timer ablk_helper cryptd pcspkr snd sg parport_pc i2c_piix4 parport video soundcore sch_fq_codel binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi ahci libahci crct10dif_pclmul crct10dif_common vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crc32c_intel serio_raw e1000 drm_panel_orientation_quirks libata dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 20 Comm: ksoftirqd/2 Tainted: G        W      ------------   3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff94ad78fa8000 ti: ffff94ad78fb0000 task.ti: ffff94ad78fb0000
RIP: 0010:[<ffffffff9b98ea15>]  [<ffffffff9b98ea15>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff94ad78fb3da8  EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffffffff9b2e2acc RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff94ad78fb3da8 R08: ffff94ae5fbf2300 R09: 00000000000500dc
R10: ffffffff9be7a380 R11: 0000000000000005 R12: 0000000025d14748
R13: ffff94aebe51b530 R14: 0000000000000246 R15: ffff94ad78fb3d70
FS:  0000000000000000(0000) GS:ffff94aebe500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001843000 CR3: 000000024cb3e000 CR4: 00000000000206e0
Call Trace:
[<ffffffff9b35ac2b>] rcu_process_callbacks+0x35b/0x570
[<ffffffff9b22b59e>] ? __switch_to+0xce/0x580
[<ffffffff9b2a4d85>] __do_softirq+0xf5/0x280
[<ffffffff9b2a4f48>] run_ksoftirqd+0x38/0x50
[<ffffffff9b2ce9e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff9b2ce8a0>] ? lg_double_unlock+0x40/0x40
[<ffffffff9b2c5f91>] kthread+0xd1/0xe0
[<ffffffff9b2c5ec0>] ? insert_kthread_work+0x40/0x40
[<ffffffff9b999ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff9b2c5ec0>] ? insert_kthread_work+0x40/0x40
Code: 36 8b 98 ff 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 e8 16 8b 98 ff 66 90 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #880587
Frame # Function Binary Source or offset Line
1
__slab_free
vmlinux 0xffffffffbc232eaa
Build id: 3.10.0-1160.81.1.el7.x86_64
0x5a
2
? rcu_process_callbacks
vmlinux 0xffffffffbc163094
Build id: 3.10.0-1160.81.1.el7.x86_64
0x224
3
kfree
vmlinux 0xffffffffbc233206
Build id: 3.10.0-1160.81.1.el7.x86_64
0x106
4
rcu_process_callbacks
vmlinux 0xffffffffbc163094
Build id: 3.10.0-1160.81.1.el7.x86_64
0x224
5
__do_softirq
vmlinux 0xffffffffbc0a9545
Build id: 3.10.0-1160.81.1.el7.x86_64
0xf5
6
run_ksoftirqd
vmlinux 0xffffffffbc0a9718
Build id: 3.10.0-1160.81.1.el7.x86_64
0x38
7
smpboot_thread_fn
vmlinux 0xffffffffbc0d42ac
Build id: 3.10.0-1160.81.1.el7.x86_64
0x14c
8
? lg_double_unlock
vmlinux 0xffffffffbc0d4160
Build id: 3.10.0-1160.81.1.el7.x86_64
0x40
9
kthread
vmlinux 0xffffffffbc0cb4d1
Build id: 3.10.0-1160.81.1.el7.x86_64
0xd1
10
? insert_kthread_work
vmlinux 0xffffffffbc0cb400
Build id: 3.10.0-1160.81.1.el7.x86_64
0x40
11
ret_from_fork_nospec_begin
vmlinux 0xffffffffbc7c51dd
Build id: 3.10.0-1160.81.1.el7.x86_64
0x7
12
? insert_kthread_work
vmlinux 0xffffffffbc0cb400
Build id: 3.10.0-1160.81.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 40s! [ksoftirqd/0:6]
Modules linked in: af_packet_diag netlink_diag vsock_diag tcp_diag udp_diag inet_diag unix_diag xt_comment xt_conntrack iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter vmw_vsock_vmci_transport vsock sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci parport_pc parport i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi scsi_transport_spi nfit mptscsih crct10dif_pclmul crct10dif_common crc32c_intel serio_raw libnvdimm mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash
dm_log dm_mod fuse
CPU: 0 PID: 6 Comm: ksoftirqd/0 Tainted: G        W  OEL ------------   3.10.0-1160.81.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff99032cbb5280 ti: ffff99032cbf8000 task.ti: ffff99032cbf8000
RIP: 0010:[<ffffffffbc232eaa>]  [<ffffffffbc232eaa>] __slab_free+0x5a/0x2b0
RSP: 0018:ffff99032cbfbce0  EFLAGS: 00010206
RAX: 0000000000000000 RBX: ffffffffbc1d204c RCX: 0000000100150015
RDX: ffff9902d9234480 RSI: fffff9b844648d00 RDI: ffff99033fc03800
RBP: ffff99032cbfbd80 R08: 0000000000000001 R09: ffffffffbc163094
R10: ffff9909ffc1f100 R11: ffff9902d9234480 R12: ffff99032cbfbcf8
R13: ffff9901c179e600 R14: 0000000000000500 R15: 000000000020001f
FS:  0000000000000000(0000) GS:ffff9909ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7f7dfc3400 CR3: 000000004aa10000 CR4: 00000000000607f0
Call Trace:
[<ffffffffbc163094>] ? rcu_process_callbacks+0x224/0x580
[<ffffffffbc233206>] kfree+0x106/0x140
[<ffffffffbc163094>] rcu_process_callbacks+0x224/0x580
[<ffffffffbc0a9545>] __do_softirq+0xf5/0x290
[<ffffffffbc0a9718>] run_ksoftirqd+0x38/0x60
[<ffffffffbc0d42ac>] smpboot_thread_fn+0x14c/0x1b0
[<ffffffffbc0d4160>] ? lg_double_unlock+0x40/0x40
[<ffffffffbc0cb4d1>] kthread+0xd1/0xe0
[<ffffffffbc0cb400>] ? insert_kthread_work+0x40/0x40
[<ffffffffbc7c51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffbc0cb400>] ? insert_kthread_work+0x40/0x40
Code: 89 54 24 08 0f 85 9a 01 00 00 66 89 5c 24 28 66 90 49 8b 4e 18 49 63 45 20 49 8b 5e 10 41 89 cc 66 44 2b 64 24 28 48 89 4c 24 48 <49> 89 1c 03 89 c8 49 8b 7d 08 c1 e8 18 41 89 c7 41 c0 ef 07 66 
Complete report #766190
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8ad89785
Build id: 3.10.0-1160.el7.x86_64
0x15
2
rcu_process_gp_end
vmlinux 0xffffffff8a759241
Build id: 3.10.0-1160.el7.x86_64
0x71
3
rcu_process_callbacks
vmlinux 0xffffffff8a75a3e5
Build id: 3.10.0-1160.el7.x86_64
0x95
4
__do_softirq
vmlinux 0xffffffff8a6a4b95
Build id: 3.10.0-1160.el7.x86_64
0xf5
5
run_ksoftirqd
vmlinux 0xffffffff8a6a4d58
Build id: 3.10.0-1160.el7.x86_64
0x38
6
smpboot_thread_fn
vmlinux 0xffffffff8a6ce674
Build id: 3.10.0-1160.el7.x86_64
0x144
7
? lg_double_unlock
vmlinux 0xffffffff8a6ce530
Build id: 3.10.0-1160.el7.x86_64
0x40
8
kthread
vmlinux 0xffffffff8a6c5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffff8a6c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffff8ad93df7
Build id: 3.10.0-1160.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffff8a6c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [ksoftirqd/2:19]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event crc32_pclmul snd_ens1371 snd_rawmidi ghash_clmulni_intel snd_ac97_codec btusb ac97_bus btrtl snd_seq btbcm btintel ppdev snd_seq_device bluetooth snd_pcm aesni_intel lrw vmw_balloon
gf128mul glue_helper ablk_helper cryptd snd_timer sg joydev snd pcspkr rfkill soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm e1000 nfit ata_piix mptspi drm scsi_transport_spi libata libnvdimm mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 19 Comm: ksoftirqd/2 Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff96ebb13ed280 ti: ffff96ebb1788000 task.ti: ffff96ebb1788000
RIP: 0010:[<ffffffff8ad89785>]  [<ffffffff8ad89785>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff96ebb178bd78  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff96ebb178bd30 RCX: 0000000000000001
RDX: ffff96ebb2c96380 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff96ebb178bd78 R08: ffff96eb9ab66000 R09: 000000000000e02d
R10: ffffffff8b27a480 R11: fffff31a80038000 R12: ffff96ebb6d28800
R13: fffffffffffffec3 R14: ffffffff8a6e1dfb R15: ffff96ebb178bd40
FS:  0000000000000000(0000) GS:ffff96ebb2c80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9102ac5390 CR3: 0000000022410000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8a759241>] rcu_process_gp_end+0x71/0x80
[<ffffffff8a75a3e5>] rcu_process_callbacks+0x95/0x570
[<ffffffff8a6a4b95>] __do_softirq+0xf5/0x280
[<ffffffff8a6a4d58>] run_ksoftirqd+0x38/0x50
[<ffffffff8a6ce674>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff8a6ce530>] ? lg_double_unlock+0x40/0x40
[<ffffffff8a6c5c21>] kthread+0xd1/0xe0
[<ffffffff8a6c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff8ad93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8a6c5b50>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #611335
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9116a175
Build id: 3.10.0-957.5.1.el7.x86_64
0x15
2
rcu_process_gp_end
vmlinux 0xffffffff90b53921
Build id: 3.10.0-957.5.1.el7.x86_64
0x71
3
rcu_process_callbacks
vmlinux 0xffffffff90b54aa5
Build id: 3.10.0-957.5.1.el7.x86_64
0x95
4
__do_softirq
vmlinux 0xffffffff90aa0f45
Build id: 3.10.0-957.5.1.el7.x86_64
0xf5
5
run_ksoftirqd
vmlinux 0xffffffff90aa1108
Build id: 3.10.0-957.5.1.el7.x86_64
0x38
6
smpboot_thread_fn
vmlinux 0xffffffff90aca604
Build id: 3.10.0-957.5.1.el7.x86_64
0x144
7
? lg_double_unlock
vmlinux 0xffffffff90aca4c0
Build id: 3.10.0-957.5.1.el7.x86_64
0x40
8
kthread
vmlinux 0xffffffff90ac1c71
Build id: 3.10.0-957.5.1.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffff90ac1ba0
Build id: 3.10.0-957.5.1.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffff91174c1d
Build id: 3.10.0-957.5.1.el7.x86_64
0x7
11
? insert_kthread_work
vmlinux 0xffffffff90ac1ba0
Build id: 3.10.0-957.5.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ksoftirqd/1:14]
Modules linked in: ext4 mbcache jbd2 uas usb_storage rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack vmhgfs(OE) ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc dm_mirror dm_region_hash dm_log dm_mod ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr snd_seq_midi snd_seq_midi_event
btusb btrtl btbcm btintel snd_ens1371 snd_rawmidi snd_ac97_codec bluetooth ac97_bus snd_seq snd_seq_device snd_pcm rfkill snd_timer snd soundcore sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci drm e1000 ata_piix mptspi libata scsi_transport_spi mptscsih mptbase nfit drm_panel_orientation_quirks libnvdimm
CPU: 1 PID: 14 Comm: ksoftirqd/1 Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-957.5.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff89573c410000 ti: ffff89573c40c000 task.ti: ffff89573c40c000
RIP: 0010:[<ffffffff9116a175>]  [<ffffffff9116a175>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff89573c40fd78  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff89573fd30400 RCX: 0000000000005927
RDX: ffff8957fd6562c0 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff89573c40fd78 R08: ffff8957bafb1e00 R09: 0000000000005929
R10: ffffffff916785c0 R11: 0000000000000000 R12: 0000000100000000
R13: ffff89573c428c18 R14: ffff8957fd65ab80 R15: 0000000100000000
FS:  0000000000000000(0000) GS:ffff8957fd640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc44fc63000 CR3: 00000000abb28000 CR4: 00000000003607e0
Call Trace:
[<ffffffff90b53921>] rcu_process_gp_end+0x71/0x80
[<ffffffff90b54aa5>] rcu_process_callbacks+0x95/0x580
[<ffffffff90aa0f45>] __do_softirq+0xf5/0x280
[<ffffffff90aa1108>] run_ksoftirqd+0x38/0x50
[<ffffffff90aca604>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff90aca4c0>] ? lg_double_unlock+0x40/0x40
[<ffffffff90ac1c71>] kthread+0xd1/0xe0
[<ffffffff90ac1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffff91174c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff90ac1ba0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #525920
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa0d89785
Build id: 3.10.0-1160.el7.x86_64
0x15
2
rcu_process_callbacks
vmlinux 0xffffffffa075a6ab
Build id: 3.10.0-1160.el7.x86_64
0x35b
3
__do_softirq
vmlinux 0xffffffffa06a4b95
Build id: 3.10.0-1160.el7.x86_64
0xf5
4
run_ksoftirqd
vmlinux 0xffffffffa06a4d58
Build id: 3.10.0-1160.el7.x86_64
0x38
5
smpboot_thread_fn
vmlinux 0xffffffffa06ce674
Build id: 3.10.0-1160.el7.x86_64
0x144
6
? lg_double_unlock
vmlinux 0xffffffffa06ce530
Build id: 3.10.0-1160.el7.x86_64
0x40
7
kthread
vmlinux 0xffffffffa06c5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffffa06c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffffa0d93df7
Build id: 3.10.0-1160.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffffa06c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [ksoftirqd/2:19]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport bnep vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul snd_ens1371 snd_rawmidi ppdev ghash_clmulni_intel snd_ac97_codec ac97_bus snd_seq snd_seq_device aesni_intel snd_pcm lrw gf128mul
btusb glue_helper vmw_balloon ablk_helper btrtl btbcm btintel cryptd bluetooth snd_timer snd joydev pcspkr soundcore rfkill sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul mptspi nfit ata_piix scsi_transport_spi drm libnvdimm crct10dif_common mptscsih crc32c_intel e1000 libata mptbase serio_raw drm_panel_orientation_quirks fuse
CPU: 2 PID: 19 Comm: ksoftirqd/2 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff88f9b7fed280 ti: ffff88f9b9b90000 task.ti: ffff88f9b9b90000
RIP: 0010:[<ffffffffa0d89785>]  [<ffffffffa0d89785>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff88f9b9b93da8  EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff88f9bb69ad40 RCX: 000000000000083f
RDX: 0000000000000001 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff88f9b9b93da8 R08: ffff88f9b9b90000 R09: 0000000000000001
R10: 0000000000000004 R11: 0000000000000293 R12: ffffffffa06e111e
R13: ffff88f9b9b93d30 R14: 0000002825cf3696 R15: ffffffffa06e43cc
FS:  0000000000000000(0000) GS:ffff88f9bb680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8dd42e5318 CR3: 0000000051066000 CR4: 0000000000360fe0
Call Trace:
[<ffffffffa075a6ab>] rcu_process_callbacks+0x35b/0x570
[<ffffffffa06a4b95>] __do_softirq+0xf5/0x280
[<ffffffffa06a4d58>] run_ksoftirqd+0x38/0x50
[<ffffffffa06ce674>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffa06ce530>] ? lg_double_unlock+0x40/0x40
[<ffffffffa06c5c21>] kthread+0xd1/0xe0
[<ffffffffa06c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffffa0d93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa06c5b50>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #480522
Frame # Function Binary Source or offset Line
1
rcu_process_callbacks
vmlinux 0xffffffff84758a06
Build id: 3.10.0-1062.4.1.el7.x86_64
0x206
2
__do_softirq
vmlinux 0xffffffff846a42c5
Build id: 3.10.0-1062.4.1.el7.x86_64
0xf5
3
run_ksoftirqd
vmlinux 0xffffffff846a4488
Build id: 3.10.0-1062.4.1.el7.x86_64
0x38
4
smpboot_thread_fn
vmlinux 0xffffffff846cdbe4
Build id: 3.10.0-1062.4.1.el7.x86_64
0x144
5
? lg_double_unlock
vmlinux 0xffffffff846cdaa0
Build id: 3.10.0-1062.4.1.el7.x86_64
0x40
6
kthread
vmlinux 0xffffffff846c51b1
Build id: 3.10.0-1062.4.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff846c50e0
Build id: 3.10.0-1062.4.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff84d8bd37
Build id: 3.10.0-1062.4.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff846c50e0
Build id: 3.10.0-1062.4.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [ksoftirqd/4:29]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13707(E) devlink binfmt_misc falcon_lsm_pinned_13601(E) ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper snd_hda_codec_generic cryptd virtio_balloon joydev snd_hda_intel sg snd_hda_codec
snd_hda_core snd_hwdep parport_pc snd_seq parport snd_seq_device snd_pcm snd_timer snd i2c_piix4 soundcore pcspkr auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul serio_raw crct10dif_common drm_panel_orientation_quirks crc32c_intel virtio_pci virtio_ring virtio floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal]
CPU: 4 PID: 29 Comm: ksoftirqd/4 Kdump: loaded Tainted: P            EL ------------   3.10.0-1062.4.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff962ef6d8a0e0 ti: ffff962ef6d98000 task.ti: ffff962ef6d98000
RIP: 0010:[<ffffffff84758a06>]  [<ffffffff84758a06>] rcu_process_callbacks+0x206/0x570
RSP: 0018:ffff962ef6d9bdb8  EFLAGS: 00010286
RAX: 0000000000000001 RBX: ffffffff85278b00 RCX: 00000001802a0022
RDX: ffff965d1c7eeb30 RSI: fffff9cc79d39c00 RDI: 0000000044042000
RBP: ffff962ef6d9be08 R08: ffff965c34e70540 R09: 00000001802a0022
R10: 0000000034e70401 R11: ffff965c34e70540 R12: 000000000000000a
R13: 0000000000000009 R14: ffff965d1c7eeb30 R15: ffff965d1f1162e0
FS:  0000000000000000(0000) GS:ffff965d1f100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000d80db004 CR3: 0000002f1ef42000 CR4: 00000000001606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff846a42c5>] __do_softirq+0xf5/0x280
[<ffffffff846a4488>] run_ksoftirqd+0x38/0x50
[<ffffffff846cdbe4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff846cdaa0>] ? lg_double_unlock+0x40/0x40
[<ffffffff846c51b1>] kthread+0xd1/0xe0
[<ffffffff846c50e0>] ? insert_kthread_work+0x40/0x40
[<ffffffff84d8bd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff846c50e0>] ? insert_kthread_work+0x40/0x40
Code: 01 4d 39 ec 7f 1d 48 8b 45 d0 48 8b 80 38 c0 ff ff a8 08 75 40 48 8b 45 c8 8b 80 a4 04 00 00 85 c0 75 32 4d 85 f6 74 2d 4c 89 f2 <48> 8b 42 08 4c 8b 32 48 3d ff 0f 00 00 41 0f 18 0e 77 b7 48 29 
Complete report #440967
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb1589785
Build id: 3.10.0-1160.el7.x86_64
0x15
2
rcu_process_callbacks
vmlinux 0xffffffffb0f5a6ab
Build id: 3.10.0-1160.el7.x86_64
0x35b
3
__do_softirq
vmlinux 0xffffffffb0ea4b95
Build id: 3.10.0-1160.el7.x86_64
0xf5
4
run_ksoftirqd
vmlinux 0xffffffffb0ea4d58
Build id: 3.10.0-1160.el7.x86_64
0x38
5
smpboot_thread_fn
vmlinux 0xffffffffb0ece674
Build id: 3.10.0-1160.el7.x86_64
0x144
6
? lg_double_unlock
vmlinux 0xffffffffb0ece530
Build id: 3.10.0-1160.el7.x86_64
0x40
7
kthread
vmlinux 0xffffffffb0ec5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffffb0ec5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffffb1593df7
Build id: 3.10.0-1160.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffffb0ec5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 33s! [ksoftirqd/3:24]
Modules linked in: veth br_netfilter tun devlink overlay(T) bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi crc32_pclmul ac97_bus ghash_clmulni_intel snd_seq snd_seq_device aesni_intel snd_pcm lrw gf128mul ppdev glue_helper ablk_helper cryptd snd_timer vmw_balloon snd soundcore joydev sg pcspkr i2c_piix4 vmw_vmci parport_pc parport xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect crc32c_intel sysimgblt fb_sys_fops ttm nfit libnvdimm drm serio_raw mptspi e1000 scsi_transport_spi ata_piix mptscsih libata mptbase drm_panel_orientation_quirks
fuse [last unloaded: ip_tables]
CPU: 3 PID: 24 Comm: ksoftirqd/3 Kdump: loaded Tainted: G    B        L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9c85b2fdb180 ti: ffff9c85b2fe8000 task.ti: ffff9c85b2fe8000
RIP: 0010:[<ffffffffb1589785>]  [<ffffffffb1589785>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9c85b2febda8  EFLAGS: 00000246
RAX: 0000000000000001 RBX: 0000000000000001 RCX: 000000000000083f
RDX: 0000000000000001 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff9c85b2febda8 R08: ffff9c85ac6de600 R09: 0000000000000001
R10: 0000000000000000 R11: 000fbf078c91e440 R12: 0000000023523f9e
R13: ffff9c85b96db530 R14: 0000000000000246 R15: ffffffffb130fd90
FS:  0000000000000000(0000) GS:ffff9c85b96c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4cbe544020 CR3: 000000012b528000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb0f5a6ab>] rcu_process_callbacks+0x35b/0x570
[<ffffffffb0ea4b95>] __do_softirq+0xf5/0x280
[<ffffffffb0ea4d58>] run_ksoftirqd+0x38/0x50
[<ffffffffb0ece674>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffb0ece530>] ? lg_double_unlock+0x40/0x40
[<ffffffffb0ec5c21>] kthread+0xd1/0xe0
[<ffffffffb0ec5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffffb1593df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb0ec5b50>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #300153
Frame # Function Binary Source or offset Line
1
dump_stack
vmlinux 0xffffffffa658311a
Build id: 3.10.0-1160.25.1.el7.x86_64
0x19
2
__warn
vmlinux 0xffffffffa5e9b1b8
Build id: 3.10.0-1160.25.1.el7.x86_64
0xd8
3
warn_slowpath_null
vmlinux 0xffffffffa5e9b2fd
Build id: 3.10.0-1160.25.1.el7.x86_64
0x1d
4
rcu_process_callbacks
vmlinux 0xffffffffa5f5ab0c
Build id: 3.10.0-1160.25.1.el7.x86_64
0x50c
5
__do_softirq
vmlinux 0xffffffffa5ea4b35
Build id: 3.10.0-1160.25.1.el7.x86_64
0xf5
6
run_ksoftirqd
vmlinux 0xffffffffa5ea4cf8
Build id: 3.10.0-1160.25.1.el7.x86_64
0x38
7
smpboot_thread_fn
vmlinux 0xffffffffa5ece7f4
Build id: 3.10.0-1160.25.1.el7.x86_64
0x144
8
? lg_double_unlock
vmlinux 0xffffffffa5ece6b0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
9
kthread
vmlinux 0xffffffffa5ec5da1
Build id: 3.10.0-1160.25.1.el7.x86_64
0xd1
10
? insert_kthread_work
vmlinux 0xffffffffa5ec5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
11
ret_from_fork_nospec_begin
vmlinux 0xffffffffa6595df7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x21
12
? insert_kthread_work
vmlinux 0xffffffffa5ec5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
WARNING: CPU: 0 PID: 6 at kernel/rcutree.c:2221 rcu_process_callbacks+0x50c/0x570
Modules linked in: veth xt_nat xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio xt_CHECKSUM loop iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter drbg ansi_cprng ccm bnep sunrpc arc4 vfat fat ath9k ath9k_common ath9k_hw ext4 mbcache ath intel_powerclamp jbd2 coretemp intel_rapl mac80211 kvm_intel r820t snd_hda_codec_realtek rtl2832 iTCO_wdt iTCO_vendor_support kvm snd_hda_codec_generic snd_hda_codec_hdmi ath3k btusb btrtl btbcm btintel bluetooth dvb_usb_rtl28xxu snd_hda_intel rtl2830 dvb_usb_v2 dvb_core snd_hda_codec irqbypass crc32_pclmul cfg80211
rc_core snd_hda_core snd_hwdep snd_seq ghash_clmulni_intel snd_seq_device sg cryptd snd_pcm pcspkr i2c_i801 rfkill mei_me snd_timer snd mei i2c_designware_platform i2c_designware_core soundcore lpc_ich ip_tables xfs libcrc32c uas usb_storage sd_mod crc_t10dif crct10dif_generic i915 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci drm libahci libata crct10dif_pclmul crct10dif_common crc32c_intel sdhci_acpi r8169 sdhci mmc_core drm_panel_orientation_quirks video iosf_mbi dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Not tainted 3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: GOOGLE Panther/Panther, BIOS MrChromebox 07/14/2017
Call Trace:
[<ffffffffa658311a>] dump_stack+0x19/0x1b
[<ffffffffa5e9b1b8>] __warn+0xd8/0x100
[<ffffffffa5e9b2fd>] warn_slowpath_null+0x1d/0x20
[<ffffffffa5f5ab0c>] rcu_process_callbacks+0x50c/0x570
[<ffffffffa5ea4b35>] __do_softirq+0xf5/0x280
[<ffffffffa5ea4cf8>] run_ksoftirqd+0x38/0x50
[<ffffffffa5ece7f4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffa5ece6b0>] ? lg_double_unlock+0x40/0x40
[<ffffffffa5ec5da1>] kthread+0xd1/0xe0
[<ffffffffa5ec5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa6595df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa5ec5cd0>] ? insert_kthread_work+0x40/0x40
Complete report #287105
Frame # Function Binary Source or offset Line
1
kfree
vmlinux 0xffffffffb2a22d45
Build id: 3.10.0-1062.el7.x86_64
0x55
2
rcu_process_callbacks
vmlinux 0xffffffffb2958954
Build id: 3.10.0-1062.el7.x86_64
0x224
3
__do_softirq
vmlinux 0xffffffffb28a41e5
Build id: 3.10.0-1062.el7.x86_64
0xf5
4
run_ksoftirqd
vmlinux 0xffffffffb28a43a8
Build id: 3.10.0-1062.el7.x86_64
0x38
5
smpboot_thread_fn
vmlinux 0xffffffffb28cdb04
Build id: 3.10.0-1062.el7.x86_64
0x144
6
? lg_double_unlock
vmlinux 0xffffffffb28cd9c0
Build id: 3.10.0-1062.el7.x86_64
0x40
7
kthread
vmlinux 0xffffffffb28c50d1
Build id: 3.10.0-1062.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffffb28c5000
Build id: 3.10.0-1062.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffffb2f8bd37
Build id: 3.10.0-1062.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffffb28c5000
Build id: 3.10.0-1062.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [ksoftirqd/0:6]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper pcspkr cryptd joydev sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi
vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel serio_raw mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Not tainted 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8ae2794a5230 ti: ffff8ae2794d8000 task.ti: ffff8ae2794d8000
RIP: 0010:[<ffffffffb2a22d45>]  [<ffffffffb2a22d45>] kfree+0x55/0x140
RSP: 0018:ffff8ae2794dbd90  EFLAGS: 00010282
RAX: ffffcd1084dd5340 RBX: ffffffffb3519200 RCX: 0000000100150013
RDX: 0000751e40000000 RSI: ffffcd1082817e00 RDI: ffff8ae27754d840
RBP: ffff8ae2794dbda8 R08: ffff8ae1e05f8cc0 R09: 0000000100150013
R10: 00000000e05f8501 R11: ffff8ae1e05f8cc0 R12: ffff8ae1e05f8cc0
R13: 00000000b28ae02c R14: ffffffffb28ae02c R15: ffff8ae1e05f8cc0
FS:  0000000000000000(0000) GS:ffff8ae27fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000558aaa192908 CR3: 0000000136d1e000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb2958954>] rcu_process_callbacks+0x224/0x570
[<ffffffffb28a41e5>] __do_softirq+0xf5/0x280
[<ffffffffb28a43a8>] run_ksoftirqd+0x38/0x50
[<ffffffffb28cdb04>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffb28cd9c0>] ? lg_double_unlock+0x40/0x40
[<ffffffffb28c50d1>] kthread+0xd1/0xe0
[<ffffffffb28c5000>] ? insert_kthread_work+0x40/0x40
[<ffffffffb2f8bd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb28c5000>] ? insert_kthread_work+0x40/0x40
Code: 8b 15 f0 52 9f 00 48 01 d8 72 0e 48 c7 c2 00 00 00 80 48 2b 15 8d 28 a2 00 48 01 d0 48 c1 e8 0c 48 c1 e0 06 48 03 05 6b 28 a2 00 <48> 8b 10 80 e6 80 0f 85 c8 00 00 00 49 89 c3 49 8b 03 a8 80 0f 
Complete report #171360
Frame # Function Binary Source or offset Line
1
dump_stack
vmlinux 0xffffffff8797ffa5
Build id: 3.10.0-1127.19.1.el7.x86_64
0x19
2
__warn
vmlinux 0xffffffff8729bd18
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd8
3
warn_slowpath_null
vmlinux 0xffffffff8729be5d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1d
4
rcu_process_callbacks
vmlinux 0xffffffff8735b16c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50c
5
__do_softirq
vmlinux 0xffffffff872a5695
Build id: 3.10.0-1127.19.1.el7.x86_64
0xf5
6
run_ksoftirqd
vmlinux 0xffffffff872a5858
Build id: 3.10.0-1127.19.1.el7.x86_64
0x38
7
smpboot_thread_fn
vmlinux 0xffffffff872cf0e4
Build id: 3.10.0-1127.19.1.el7.x86_64
0x144
8
? lg_double_unlock
vmlinux 0xffffffff872cefa0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
9
kthread
vmlinux 0xffffffff872c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
10
? insert_kthread_work
vmlinux 0xffffffff872c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
11
ret_from_fork_nospec_begin
vmlinux 0xffffffff87992d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
12
? insert_kthread_work
vmlinux 0xffffffff872c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
WARNING: CPU: 0 PID: 6 at kernel/rcutree.c:2221 rcu_process_callbacks+0x50c/0x570
Modules linked in: r820t rtl2832 dvb_usb_rtl28xxu rtl2830 dvb_usb_v2 dvb_core rc_core xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) sunrpc squashfs loop dm_mirror dm_region_hash dm_log dm_mod usblp snd_hda_codec_idt snd_hda_codec_generic joydev snd_hda_intel iTCO_wdt snd_hda_codec iTCO_vendor_support
gpio_ich ppdev snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm sg intel_powerclamp pcspkr i2c_i801 lpc_ich snd_timer snd soundcore parport_pc parport acpi_cpufreq fuse binfmt_misc ip_tables ext4 mbcache jbd2 radeon sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_generic pata_acpi drm e1000e ata_piix libata firewire_ohci firewire_core ptp serio_raw pps_core crc_itu_t drm_panel_orientation_quirks
CPU: 0 PID: 6 Comm: ksoftirqd/0 Tainted: G           OE  ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name:                  /D945PSN, BIOS SN94510J.86A.0108.2008.0817.1811 08/17/2008
Call Trace:
[<ffffffff8797ffa5>] dump_stack+0x19/0x1b
[<ffffffff8729bd18>] __warn+0xd8/0x100
[<ffffffff8729be5d>] warn_slowpath_null+0x1d/0x20
[<ffffffff8735b16c>] rcu_process_callbacks+0x50c/0x570
[<ffffffff872a5695>] __do_softirq+0xf5/0x280
[<ffffffff872a5858>] run_ksoftirqd+0x38/0x50
[<ffffffff872cf0e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff872cefa0>] ? lg_double_unlock+0x40/0x40
[<ffffffff872c6691>] kthread+0xd1/0xe0
[<ffffffff872c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff87992d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff872c65c0>] ? insert_kthread_work+0x40/0x40
Complete report #115276
Frame # Function Binary Source or offset Line
1
rcu_process_callbacks
vmlinux 0xffffffff8f15afb6
Build id: 3.10.0-1127.el7.x86_64
0x2f6
2
__do_softirq
vmlinux 0xffffffff8f0a5695
Build id: 3.10.0-1127.el7.x86_64
0xf5
3
run_ksoftirqd
vmlinux 0xffffffff8f0a5858
Build id: 3.10.0-1127.el7.x86_64
0x38
4
smpboot_thread_fn
vmlinux 0xffffffff8f0cf0e4
Build id: 3.10.0-1127.el7.x86_64
0x144
5
? lg_double_unlock
vmlinux 0xffffffff8f0cefa0
Build id: 3.10.0-1127.el7.x86_64
0x40
6
kthread
vmlinux 0xffffffff8f0c6691
Build id: 3.10.0-1127.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff8f0c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff8f792d37
Build id: 3.10.0-1127.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff8f0c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ksoftirqd/0:6]
Modules linked in: qingteng(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock iosf_mbi snd_seq_midi kvm_intel snd_seq_midi_event kvm irqbypass crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq aesni_intel lrw snd_seq_device gf128mul ppdev glue_helper ablk_helper snd_pcm vmw_balloon cryptd snd_timer pcspkr joydev snd soundcore sg parport_pc
parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif ata_generic pata_acpi vmwgfx crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul crct10dif_common mptspi crc32c_intel scsi_transport_spi drm nfit e1000 serio_raw mptscsih mptbase libata libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9b74ba635230 ti: ffff9b74ba6f4000 task.ti: ffff9b74ba6f4000
RIP: 0010:[<ffffffff8f15afb6>]  [<ffffffff8f15afb6>] rcu_process_callbacks+0x2f6/0x570
RSP: 0018:ffff9b74ba6f7db8  EFLAGS: 00000206
RAX: ffffffffffffd800 RBX: 0000000000000002 RCX: ffff9b7441dd6750
RDX: 0000000000000200 RSI: fffff72ac1e18200 RDI: 0000000000000206
RBP: ffff9b74ba6f7e08 R08: ffff9b74b8609200 R09: 0000000180400026
R10: 00000000b860bc01 R11: ffff9b74b8609200 R12: ffff9b74ba6f7d58
R13: 0000000180400026 R14: ffff9b74bffd9800 R15: fffff72ac1a4edc0
FS:  0000000000000000(0000) GS:ffff9b74bb600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd2d5872000 CR3: 0000000069882000 CR4: 00000000003607f0
Call Trace:
[<ffffffff8f0a5695>] __do_softirq+0xf5/0x280
[<ffffffff8f0a5858>] run_ksoftirqd+0x38/0x50
[<ffffffff8f0cf0e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff8f0cefa0>] ? lg_double_unlock+0x40/0x40
[<ffffffff8f0c6691>] kthread+0xd1/0xe0
[<ffffffff8f0c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8f792d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8f0c65c0>] ? insert_kthread_work+0x40/0x40
Code: 21 aa b0 00 48 39 c2 7d 07 49 89 97 80 00 00 00 48 85 d2 0f 94 c2 49 83 7f 28 00 0f 94 c0 38 c2 0f 85 fd 01 00 00 4c 89 e7 57 9d <0f> 1f 44 00 00 48 8b 45 c0 49 3b 47 30 74 0c 49 83 7f 48 00 74 
Complete report #91167
Frame # Function Binary Source or offset Line
1
free_fib_info_rcu
vmlinux 0xffffffff8a8e554e
Build id: 3.10.0-1160.11.1.el7.x86_64
0x17e
2
rcu_process_callbacks
vmlinux 0xffffffff8a35a788
Build id: 3.10.0-1160.11.1.el7.x86_64
0x1d8
3
__do_softirq
vmlinux 0xffffffff8a2a4b95
Build id: 3.10.0-1160.11.1.el7.x86_64
0xf5
4
run_ksoftirqd
vmlinux 0xffffffff8a2a4d58
Build id: 3.10.0-1160.11.1.el7.x86_64
0x38
5
smpboot_thread_fn
vmlinux 0xffffffff8a2ce8c4
Build id: 3.10.0-1160.11.1.el7.x86_64
0x144
6
? lg_double_unlock
vmlinux 0xffffffff8a2ce780
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
7
kthread
vmlinux 0xffffffff8a2c5e71
Build id: 3.10.0-1160.11.1.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffff8a2c5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff8a993df7
Build id: 3.10.0-1160.11.1.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffff8a2c5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [ksoftirqd/7:44]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 iosf_mbi snd_rawmidi crc32_pclmul snd_ac97_codec ghash_clmulni_intel ac97_bus aesni_intel snd_seq lrw gf128mul snd_seq_device snd_pcm glue_helper snd_timer ablk_helper cryptd snd soundcore ppdev sg pcspkr joydev vmw_balloon parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic
pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit crct10dif_pclmul crct10dif_common mptspi ata_piix libnvdimm crc32c_intel drm scsi_transport_spi libata mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 7 PID: 44 Comm: ksoftirqd/7 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff92e8b7f4a100 ti: ffff92e8b7f64000 task.ti: ffff92e8b7f64000
RIP: 0010:[<ffffffff8a8e554e>]  [<ffffffff8a8e554e>] free_fib_info_rcu+0x17e/0x250
RSP: 0018:ffff92e8b7f67d68  EFLAGS: 00010287
RAX: ffff92ef5ea80000 RBX: ffff92eed1843800 RCX: 0000000000000012
RDX: 0000000000000080 RSI: 0000000000000040 RDI: 0000000000000040
RBP: ffff92e8b7f67da8 R08: ffffffff8af5e108 R09: 0000000100400013
R10: 00000000d1843c01 R11: ffff92eed1843800 R12: 0000000000000000
R13: ffffffff8a425fed R14: ffff92e8b7f67d70 R15: 0000000100400013
FS:  0000000000000000(0000) GS:ffff92ef5d7c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00084d000 CR3: 00000005ec610000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8a35a788>] rcu_process_callbacks+0x1d8/0x570
[<ffffffff8a2a4b95>] __do_softirq+0xf5/0x280
[<ffffffff8a2a4d58>] run_ksoftirqd+0x38/0x50
[<ffffffff8a2ce8c4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff8a2ce780>] ? lg_double_unlock+0x40/0x40
[<ffffffff8a2c5e71>] kthread+0xd1/0xe0
[<ffffffff8a2c5da0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8a993df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8a2c5da0>] ? insert_kthread_work+0x40/0x40
Code: 24 01 48 8b 3d 84 dd 12 00 48 63 f0 48 63 d2 e8 39 79 cb ff 8b 15 c7 9f 67 00 41 89 c4 39 d0 7d 43 48 98 48 8b 04 c5 e0 13 f5 8a <48> 8b 3c 03 89 d0 48 85 ff 74 c7 66 83 7f 66 00 7f c0 8b 97 80 
Complete report #80711
Frame # Function Binary Source or offset Line
1
rcu_process_callbacks
vmlinux 0xffffffffb055ae66
Build id: 3.10.0-1127.18.2.el7.x86_64
0x206
2
__do_softirq
vmlinux 0xffffffffb04a5695
Build id: 3.10.0-1127.18.2.el7.x86_64
0xf5
3
run_ksoftirqd
vmlinux 0xffffffffb04a5858
Build id: 3.10.0-1127.18.2.el7.x86_64
0x38
4
smpboot_thread_fn
vmlinux 0xffffffffb04cf0e4
Build id: 3.10.0-1127.18.2.el7.x86_64
0x144
5
? lg_double_unlock
vmlinux 0xffffffffb04cefa0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x40
6
kthread
vmlinux 0xffffffffb04c6691
Build id: 3.10.0-1127.18.2.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffb04c65c0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffb0b92d1d
Build id: 3.10.0-1127.18.2.el7.x86_64
0x7
9
? insert_kthread_work
vmlinux 0xffffffffb04c65c0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 29s! [ksoftirqd/5:34]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink binfmt_misc sunrpc nfit libnvdimm ppdev iosf_mbi sg parport_pc i2c_piix4 parport crc32_pclmul ghash_clmulni_intel joydev virtio_balloon aesni_intel pcspkr lrw gf128mul glue_helper ablk_helper cryptd ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix virtio_console virtio_blk libata 8139too crct10dif_pclmul crct10dif_common serio_raw drm_panel_orientation_quirks virtio_pci floppy crc32c_intel 8139cp
virtio_ring virtio mii dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 34 Comm: ksoftirqd/5 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1127.18.2.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff966537428000 ti: ffff966537430000 task.ti: ffff966537430000
RIP: 0010:[<ffffffffb055ae66>]  [<ffffffffb055ae66>] rcu_process_callbacks+0x206/0x570
RSP: 0018:ffff966537433db8  EFLAGS: 00010286
RAX: 0000000000000001 RBX: ffffffffb107a280 RCX: 000000018020001c
RDX: ffff9663f5d39400 RSI: fffffb8945c88300 RDI: 0000000040000000
RBP: ffff966537433e08 R08: ffff96653220c700 R09: 000000018020001c
R10: 000000003220df01 R11: ffff96653220c700 R12: 000000000000000a
R13: 0000000000000004 R14: ffff9663f5d39400 R15: ffff96653a296380
FS:  0000000000000000(0000) GS:ffff96653a280000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000001bed0005 CR3: 0000000090c10000 CR4: 00000000007606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffb04a5695>] __do_softirq+0xf5/0x280
[<ffffffffb04a5858>] run_ksoftirqd+0x38/0x50
[<ffffffffb04cf0e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffb04cefa0>] ? lg_double_unlock+0x40/0x40
[<ffffffffb04c6691>] kthread+0xd1/0xe0
[<ffffffffb04c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb0b92d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffb04c65c0>] ? insert_kthread_work+0x40/0x40
Code: 01 4d 39 ec 7f 1d 48 8b 45 d0 48 8b 80 38 c0 ff ff a8 08 75 40 48 8b 45 c8 8b 80 a4 04 00 00 85 c0 75 32 4d 85 f6 74 2d 4c 89 f2 <48> 8b 42 08 4c 8b 32 48 3d ff 0f 00 00 41 0f 18 0e 77 b7 48 29 
Complete report #13883
Frame # Function Binary Source or offset Line
1
dump_stack
vmlinux 0xffffffff8a17ffa5
Build id: 3.10.0-1127.19.1.el7.x86_64
0x19
2
__warn
vmlinux 0xffffffff89a9bd18
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd8
3
warn_slowpath_null
vmlinux 0xffffffff89a9be5d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1d
4
rcu_process_callbacks
vmlinux 0xffffffff89b5b16c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50c
5
__do_softirq
vmlinux 0xffffffff89aa5695
Build id: 3.10.0-1127.19.1.el7.x86_64
0xf5
6
run_ksoftirqd
vmlinux 0xffffffff89aa5858
Build id: 3.10.0-1127.19.1.el7.x86_64
0x38
7
smpboot_thread_fn
vmlinux 0xffffffff89acf0e4
Build id: 3.10.0-1127.19.1.el7.x86_64
0x144
8
? lg_double_unlock
vmlinux 0xffffffff89acefa0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
9
kthread
vmlinux 0xffffffff89ac6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
10
? insert_kthread_work
vmlinux 0xffffffff89ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
11
ret_from_fork_nospec_begin
vmlinux 0xffffffff8a192d1d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7
12
? insert_kthread_work
vmlinux 0xffffffff89ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
WARNING: CPU: 1 PID: 14 at kernel/rcutree.c:2221 rcu_process_callbacks+0x50c/0x570
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc vfat fat radeon intel_pmc_core eeepc_wmi asus_wmi intel_powerclamp iTCO_wdt sparse_keymap coretemp iTCO_vendor_support intel_rapl rfkill iosf_mbi snd_hda_codec_realtek snd_hda_codec_hdmi mxm_wmi snd_hda_codec_generic kvm snd_hda_intel snd_hda_codec i2c_algo_bit ttm
snd_hda_core drm_kms_helper snd_hwdep irqbypass snd_seq snd_seq_device syscopyarea crc32_pclmul sysfillrect sysimgblt fb_sys_fops ghash_clmulni_intel snd_pcm joydev drm aesni_intel lrw snd_timer gf128mul glue_helper ablk_helper cryptd snd sg mei_me pcspkr i2c_i801 soundcore mei drm_panel_orientation_quirks pinctrl_sunrisepoint pinctrl_intel video wmi acpi_pad ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic hid_logitech_dj ahci libahci libata crct10dif_pclmul crct10dif_common r8169 crc32c_intel serio_raw i2c_hid dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 14 Comm: ksoftirqd/1 Kdump: loaded Tainted: G        W      ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: System manufacturer System Product Name/H110I-PLUS, BIOS 3805 05/07/2018
Call Trace:
[<ffffffff8a17ffa5>] dump_stack+0x19/0x1b
[<ffffffff89a9bd18>] __warn+0xd8/0x100
[<ffffffff89a9be5d>] warn_slowpath_null+0x1d/0x20
[<ffffffff89b5b16c>] rcu_process_callbacks+0x50c/0x570
[<ffffffff89aa5695>] __do_softirq+0xf5/0x280
[<ffffffff89aa5858>] run_ksoftirqd+0x38/0x50
[<ffffffff89acf0e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff89acefa0>] ? lg_double_unlock+0x40/0x40
[<ffffffff89ac6691>] kthread+0xd1/0xe0
[<ffffffff89ac65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8a192d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff89ac65c0>] ? insert_kthread_work+0x40/0x40
Complete report #3825
Frame # Function Binary Source or offset Line
1
rcu_process_callbacks
vmlinux 0xffffffffb8554e4e
Build id: 3.10.0-957.12.2.el7.x86_64
0x20e
2
__do_softirq
vmlinux 0xffffffffb84a0ff5
Build id: 3.10.0-957.12.2.el7.x86_64
0xf5
3
run_ksoftirqd
vmlinux 0xffffffffb84a11b8
Build id: 3.10.0-957.12.2.el7.x86_64
0x38
4
smpboot_thread_fn
vmlinux 0xffffffffb84ca6b4
Build id: 3.10.0-957.12.2.el7.x86_64
0x144
5
? lg_double_unlock
vmlinux 0xffffffffb84ca570
Build id: 3.10.0-957.12.2.el7.x86_64
0x40
6
kthread
vmlinux 0xffffffffb84c1d21
Build id: 3.10.0-957.12.2.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffb84c1c50
Build id: 3.10.0-957.12.2.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffb8b75c37
Build id: 3.10.0-957.12.2.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffb84c1c50
Build id: 3.10.0-957.12.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [ksoftirqd/2:19]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack binfmt_misc ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac coretemp ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel
serio_raw ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm vmxnet3 mptspi libata scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 19 Comm: ksoftirqd/2 Kdump: loaded Tainted: G             L ------------   3.10.0-957.12.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9eb893d28000 ti: ffff9eb893d24000 task.ti: ffff9eb893d24000
RIP: 0010:[<ffffffffb8554e4e>]  [<ffffffffb8554e4e>] rcu_process_callbacks+0x20e/0x580
RSP: 0018:ffff9eb893d27db8  EFLAGS: 00010282
RAX: 0000000000000000 RBX: ffffffffb9117840 RCX: 000000000001fa80
RDX: ffff9ec7fe0c9cb0 RSI: 0000000000000000 RDI: 0000000000000001
RBP: ffff9eb893d27e08 R08: ffff9ed64f724290 R09: 0000000180180017
R10: 0000000000000001 R11: fffff463bc3dc900 R12: ffff9ed64dafbb00
R13: 0000000180150010 R14: ffffffffb84dcc3e R15: 0000000180180017
FS:  0000000000000000(0000) GS:ffff9ed6cfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe7a1401df0 CR3: 00000010ddb88000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb84a0ff5>] __do_softirq+0xf5/0x280
[<ffffffffb84a11b8>] run_ksoftirqd+0x38/0x50
[<ffffffffb84ca6b4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffb84ca570>] ? lg_double_unlock+0x40/0x40
[<ffffffffb84c1d21>] kthread+0xd1/0xe0
[<ffffffffb84c1c50>] ? insert_kthread_work+0x40/0x40
[<ffffffffb8b75c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb84c1c50>] ? insert_kthread_work+0x40/0x40
Code: 01 4d 39 ec 7f 1d 48 8b 45 d0 48 8b 80 38 c0 ff ff a8 08 75 40 48 8b 45 c8 8b 80 a4 04 00 00 85 c0 75 32 4d 85 f6 74 2d 4c 89 f2 <48> 8b 42 08 4c 8b 32 48 3d ff 0f 00 00 41 0f 18 0e 77 b7 48 29