Complete report #947667
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
native_queued_spin_lock_slowpath
|
vmlinux |
0xffffffff82d12288
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x158
|
|
2
|
queued_spin_lock_slowpath
|
vmlinux |
0xffffffff8335d1ab
Build id: 3.10.0-957.12.1.el7.x86_64
|
0xb
|
|
3
|
_raw_spin_lock
|
vmlinux |
0xffffffff8336b660
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x20
|
|
4
|
__ext4_es_shrink
|
ext4 |
0xffffffffc0847adb
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x5b
|
|
5
|
ext4_es_shrink
|
ext4 |
0xffffffffc0847dd4
Build id: 3.10.0-957.12.1.el7.x86_64
|
0xb4
|
|
6
|
shrink_slab
|
vmlinux |
0xffffffff82dc9e85
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x175
|
|
7
|
|
vmlinux |
0xffffffff82e37767
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x87
|
|
8
|
balance_pgdat
|
vmlinux |
0xffffffff82dcdc81
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x4b1
|
|
9
|
kswapd
|
vmlinux |
0xffffffff82dcdf23
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x173
|
|
10
|
|
vmlinux |
0xffffffff82cc2d40
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x30
|
|
11
|
|
vmlinux |
0xffffffff82dcddb0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x5e0
|
|
12
|
kthread
|
vmlinux |
0xffffffff82cc1c71
Build id: 3.10.0-957.12.1.el7.x86_64
|
0xd1
|
|
13
|
|
vmlinux |
0xffffffff82cc1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x40
|
|
14
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff83375bdd
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x7
|
|
15
|
|
vmlinux |
0xffffffff82cc1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#26 stuck for 22s! [kswapd0:350]
Modules linked in: ebt_arp vhost_net vhost macvtap macvlan tun drbd_transport_tcp(OE) drbd(OE) iscsi_tcp libiscsi_tcp libiscsi fuse udp_diag tcp_diag inet_diag nf_conntrack_netlink geneve binfmt_misc devlink bnx2fc cnic uio fcoe libfcoe libfc scsi_transport_fc scsi_tgt scsi_transport_iscsi ip6t_MASQUERADE nf_nat_masquerade_ipv6 ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack openvswitch ip_set nfnetlink ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables
ip6table_filter ip6_tables vxlan ip6_udp_tunnel udp_tunnel iptable_filter sunrpc vfat fat xfs libcrc32c iTCO_wdt iTCO_vendor_support dell_smbios dcdbas dell_wmi_descriptor skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_service_time dm_multipath pcspkr joydev sg lpc_ich i2c_i801 mei_me mei wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad tpm_crb ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common mgag200 crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nvme ahci i40e nvme_core igb libahci megaraid_sas libata drm_panel_orientation_quirks ptp pps_core dca i2c_algo_bit nfit
libnvdimm dm_mirror dm_region_hash dm_log dm_mod 8021q garp mrp bridge stp llc bonding dummy
CPU: 26 PID: 350 Comm: kswapd0 Kdump: loaded Tainted: G OEL ------------ 3.10.0-957.12.1.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R640/0HG0J8, BIOS 2.13.3 12/13/2021
task: ffff98973b3e5140 ti: ffff98673b988000 task.ti: ffff98673b988000
RIP: 0010:[<ffffffff82d12288>] [<ffffffff82d12288>] native_queued_spin_lock_slowpath+0x158/0x200
RSP: 0018:ffff98673b98bbc0 EFLAGS: 00000202
RAX: 0000000000000101 RBX: ffff98673b98bb98 RCX: 0000000000d10000
RDX: 0000000001610101 RSI: 0000000000000101 RDI: ffff989732fad480
RBP: ffff98673b98bbc0 R08: ffff98673dd5b780 R09: 0000000000000000
R10: 0000000000000307 R11: 7fffffffffffffff R12: 000000000000001a
R13: ffffb332804fe8f0 R14: ffff9858a12d4080 R15: 0000000006424a54
FS: 0000000000000000(0000) GS:ffff98673dd40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002b4eff249000 CR3: 000000325c010000 CR4: 00000000007627e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffff8335d1ab>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff8336b660>] _raw_spin_lock+0x20/0x30
[<ffffffffc0847adb>] __ext4_es_shrink+0x5b/0x2a0 [ext4]
[<ffffffffc0847dd4>] ext4_es_shrink+0xb4/0x130 [ext4]
[<ffffffff82dc9e85>] shrink_slab+0x175/0x340
[<ffffffff82e37767>] ? vmpressure+0x87/0x90
[<ffffffff82dcdc81>] balance_pgdat+0x4b1/0x5e0
[<ffffffff82dcdf23>] kswapd+0x173/0x440
[<ffffffff82cc2d40>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff82dcddb0>] ? balance_pgdat+0x5e0/0x5e0
[<ffffffff82cc1c71>] kthread+0xd1/0xe0
[<ffffffff82cc1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffff83375bdd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff82cc1ba0>] ? insert_kthread_work+0x40/0x40
Complete report #576383
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
native_queued_spin_lock_slowpath
|
vmlinux |
0xffffffff8311757d
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x1d
|
|
2
|
queued_spin_lock_slowpath
|
vmlinux |
0xffffffff837754ee
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0xb
|
|
3
|
_raw_spin_lock
|
vmlinux |
0xffffffff83783b20
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x20
|
|
4
|
__ext4_es_shrink
|
ext4 |
0xffffffffc09ad0bb
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x5b
|
|
5
|
|
vmlinux |
0xffffffff8326996c
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x17c
|
|
6
|
ext4_es_shrink
|
ext4 |
0xffffffffc09ad3b4
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0xb4
|
|
7
|
shrink_slab
|
vmlinux |
0xffffffff831d1175
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x175
|
|
8
|
|
vmlinux |
0xffffffff832405f1
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x61
|
|
9
|
balance_pgdat
|
vmlinux |
0xffffffff831d4ea8
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x3a8
|
|
10
|
kswapd
|
vmlinux |
0xffffffff831d5253
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x173
|
|
11
|
|
vmlinux |
0xffffffff830c7410
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x30
|
|
12
|
|
vmlinux |
0xffffffff831d50e0
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x5e0
|
|
13
|
kthread
|
vmlinux |
0xffffffff830c6321
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0xd1
|
|
14
|
|
vmlinux |
0xffffffff830c6250
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x40
|
|
15
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff8378dd37
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x21
|
|
16
|
|
vmlinux |
0xffffffff830c6250
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kswapd0:189]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_14203(E) falcon_lsm_pinned_13804(E) falcon_lsm_pinned_13501(E) falcon_lsm_pinned_13109(E) falcon_lsm_pinned_13005(E) falcon_lsm_pinned_12905(E) binfmt_misc cmac arc4 md4 nls_utf8 cifs ccm dns_resolver ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_comment xt_mark iptable_filter iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_netlink nfnetlink iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi overlay(T) vmw_vsock_vmci_transport vsock falcon_lsm_pinned_12803(E) ext4 sb_edac mbcache iosf_mbi jbd2 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper pcc_cpufreq cryptd ppdev vmw_balloon
pcspkr joydev parport_pc parport vmw_vmci sg i2c_piix4 br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi nfit crct10dif_pclmul libata crct10dif_common crc32c_intel mptscsih libnvdimm serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal]
CPU: 0 PID: 189 Comm: kswapd0 Kdump: loaded Tainted: P EL ------------ T 3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff89133e8ae2a0 ti: ffff89033d214000 task.ti: ffff89033d214000
RIP: 0010:[<ffffffff8311757d>] [<ffffffff8311757d>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff89033d217bc0 EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000246 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff891332f19c80
RBP: ffff89033d217bc0 R08: 000000000000da10 R09: 00000000001e35fc
R10: 0000000000000001 R11: ffff8910957d7f00 R12: 0000000000000001
R13: ffff8910957d7f00 R14: dead000000000200 R15: ffff89033d217bc8
FS: 0000000000000000(0000) GS:ffff89033fa00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0009162e0 CR3: 0000000169810000 CR4: 00000000001607f0
Call Trace:
[<ffffffff837754ee>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff83783b20>] _raw_spin_lock+0x20/0x30
[<ffffffffc09ad0bb>] __ext4_es_shrink+0x5b/0x2a0 [ext4]
[<ffffffff8326996c>] ? prune_icache_sb+0x17c/0x330
[<ffffffffc09ad3b4>] ext4_es_shrink+0xb4/0x130 [ext4]
[<ffffffff831d1175>] shrink_slab+0x175/0x340
[<ffffffff832405f1>] ? vmpressure+0x61/0x90
[<ffffffff831d4ea8>] balance_pgdat+0x3a8/0x5e0
[<ffffffff831d5253>] kswapd+0x173/0x430
[<ffffffff830c7410>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff831d50e0>] ? balance_pgdat+0x5e0/0x5e0
[<ffffffff830c6321>] kthread+0xd1/0xe0
[<ffffffff830c6250>] ? insert_kthread_work+0x40/0x40
[<ffffffff8378dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff830c6250>] ? insert_kthread_work+0x40/0x40
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f
Complete report #511597
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
native_queued_spin_lock_slowpath
|
vmlinux |
0xffffffff810fa322
Build id: 3.10.0-693.el7.x86_64
|
0x112
|
|
2
|
queued_spin_lock_slowpath
|
vmlinux |
0xffffffff8169e5ff
Build id: 3.10.0-693.el7.x86_64
|
0xb
|
|
3
|
_raw_spin_lock
|
vmlinux |
0xffffffff816abb50
Build id: 3.10.0-693.el7.x86_64
|
0x20
|
|
4
|
__ext4_es_shrink
|
ext4 |
0xffffffffc09987eb
Build id: 3.10.0-693.el7.x86_64
|
0x5b
|
|
5
|
ext4_es_shrink
|
ext4 |
0xffffffffc0998ae2
Build id: 3.10.0-693.el7.x86_64
|
0xb2
|
|
6
|
shrink_slab
|
vmlinux |
0xffffffff81195413
Build id: 3.10.0-693.el7.x86_64
|
0x163
|
|
7
|
|
vmlinux |
0xffffffff811f7511
Build id: 3.10.0-693.el7.x86_64
|
0x61
|
|
8
|
balance_pgdat
|
vmlinux |
0xffffffff81199081
Build id: 3.10.0-693.el7.x86_64
|
0x4b1
|
|
9
|
kswapd
|
vmlinux |
0xffffffff81199323
Build id: 3.10.0-693.el7.x86_64
|
0x173
|
|
10
|
|
vmlinux |
0xffffffff810b1910
Build id: 3.10.0-693.el7.x86_64
|
0x30
|
|
11
|
|
vmlinux |
0xffffffff811991b0
Build id: 3.10.0-693.el7.x86_64
|
0x5e0
|
|
12
|
kthread
|
vmlinux |
0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64
|
0xcf
|
|
13
|
|
vmlinux |
0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
|
0x40
|
|
14
|
ret_from_fork
|
vmlinux |
0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64
|
0x58
|
|
15
|
|
vmlinux |
0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#120 stuck for 23s! [kswapd0:990]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi iTCO_wdt iTCO_vendor_support dell_smbios dcdbas sparse_keymap kvm_intel ext4 mbcache jbd2 kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev ipmi_ssif dm_service_time
qla2xxx ipmi_si pcspkr ipmi_devintf sg ipmi_msghandler wmi acpi_power_meter i2c_i801 mei_me nfit shpchp mei libnvdimm lpc_ich acpi_pad dm_multipath nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc32c_intel lpfc mgag200 i2c_algo_bit drm_kms_helper megaraid_sas syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc_t10dif crct10dif_generic crct10dif_pclmul drm tg3 scsi_transport_fc ahci libahci scsi_tgt crct10dif_common ptp libata pps_core i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 120 PID: 990 Comm: kswapd0 Not tainted 3.10.0-693.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R940xa/08XR9M, BIOS 1.3.8 11/21/2018
task: ffff887ffd50cf10 ti: ffff881ffdbb4000 task.ti: ffff881ffdbb4000
RIP: 0010:[<ffffffff810fa322>] [<ffffffff810fa322>] native_queued_spin_lock_slowpath+0x112/0x1e0
RSP: 0018:ffff881ffdbb7bc0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8834123d8c18 RCX: 0000000003c10000
RDX: ffff883ffe317880 RSI: 0000000000890101 RDI: ffff885fe1f99480
RBP: ffff881ffdbb7bc0 R08: ffff881ffed97880 R09: 0000000000000000
R10: 00000000e7a85101 R11: ffffea00f79ea140 R12: ffff885ffab74400
R13: ffff883de7a85208 R14: ffff8817f4d2d810 R15: 00000000b8b73b45
FS: 0000000000000000(0000) GS:ffff881ffed80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffc33f95fc0 CR3: 00000000019f2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff881ffdbb7bd0 ffffffff8169e5ff ffff881ffdbb7be0 ffffffff816abb50
ffff881ffdbb7c58 ffffffffc09987eb ffff885fe1f99480 0000000100000080
0000000000000000 ffff883400000001 0001cb0e00000080 ffff881ffdbb7c18
Call Trace:
[<ffffffff8169e5ff>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff816abb50>] _raw_spin_lock+0x20/0x30
[<ffffffffc09987eb>] __ext4_es_shrink+0x5b/0x2a0 [ext4]
[<ffffffffc0998ae2>] ext4_es_shrink+0xb2/0x130 [ext4]
[<ffffffff81195413>] shrink_slab+0x163/0x330
[<ffffffff811f7511>] ? vmpressure+0x61/0x90
[<ffffffff81199081>] balance_pgdat+0x4b1/0x5e0
[<ffffffff81199323>] kswapd+0x173/0x440
[<ffffffff810b1910>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811991b0>] ? balance_pgdat+0x5e0/0x5e0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 13 48 c1 ea 0d 48 98 83 e2 30 48 81 c2 80 78 01 00 48 03 14 c5 e0 fd b0 81 4c 89 02 41 8b 40 08 85 c0 75 0f 0f 1f 44 00 00 f3 90 <41> 8b 40 08 85 c0 74 f6 4d 8b 08 4d 85 c9 74 04 41 0f 18 09 8b
Complete report #401715
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
native_queued_spin_lock_slowpath
|
vmlinux |
0xffffffff810fa3ce
Build id: 3.10.0-693.el7.x86_64
|
0x1be
|
|
2
|
queued_spin_lock_slowpath
|
vmlinux |
0xffffffff8169e5ff
Build id: 3.10.0-693.el7.x86_64
|
0xb
|
|
3
|
_raw_spin_lock
|
vmlinux |
0xffffffff816abb50
Build id: 3.10.0-693.el7.x86_64
|
0x20
|
|
4
|
__ext4_es_shrink
|
ext4 |
0xffffffffc07b17eb
Build id: 3.10.0-693.el7.x86_64
|
0x5b
|
|
5
|
|
vmlinux |
0xffffffff8121e724
Build id: 3.10.0-693.el7.x86_64
|
0x174
|
|
6
|
ext4_es_shrink
|
ext4 |
0xffffffffc07b1ae2
Build id: 3.10.0-693.el7.x86_64
|
0xb2
|
|
7
|
shrink_slab
|
vmlinux |
0xffffffff81195413
Build id: 3.10.0-693.el7.x86_64
|
0x163
|
|
8
|
|
vmlinux |
0xffffffff811f7537
Build id: 3.10.0-693.el7.x86_64
|
0x87
|
|
9
|
balance_pgdat
|
vmlinux |
0xffffffff81199081
Build id: 3.10.0-693.el7.x86_64
|
0x4b1
|
|
10
|
kswapd
|
vmlinux |
0xffffffff81199323
Build id: 3.10.0-693.el7.x86_64
|
0x173
|
|
11
|
|
vmlinux |
0xffffffff810b1910
Build id: 3.10.0-693.el7.x86_64
|
0x30
|
|
12
|
|
vmlinux |
0xffffffff811991b0
Build id: 3.10.0-693.el7.x86_64
|
0x5e0
|
|
13
|
kthread
|
vmlinux |
0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64
|
0xcf
|
|
14
|
|
vmlinux |
0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
|
0x40
|
|
15
|
ret_from_fork
|
vmlinux |
0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64
|
0x58
|
|
16
|
|
vmlinux |
0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#37 stuck for 22s! [kswapd1:507]
Modules linked in: binfmt_misc l2tp_ppp l2tp_netlink l2tp_core pppox ppp_generic slhc xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ext4 mbcache jbd2 dell_smbios sparse_keymap iTCO_wdt iTCO_vendor_support dcdbas skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi nfit libnvdimm kvm_intel kvm ipmi_ssif irqbypass joydev i2c_i801 pcspkr sg lpc_ich
shpchp ipmi_si ipmi_devintf ipmi_msghandler mei_me mei acpi_power_meter acpi_pad fuse ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd 8021q scsi_transport_iscsi mgag200 garp stp llc drm_kms_helper mrp syscopyarea sysfillrect sysimgblt fb_sys_fops ttm igb drm ahci dca libahci ptp pps_core libata i2c_algo_bit megaraid_sas bnxt_en i2c_core wmi sunrpc dm_mirror dm_region_hash dm_log dm_mod
CPU: 37 PID: 507 Comm: kswapd1 Tainted: G L ------------ 3.10.0-693.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge T640/07978V, BIOS 2.9.4 11/09/2020
task: ffff887f68365ee0 ti: ffff883f685c8000 task.ti: ffff883f685c8000
RIP: 0010:[<ffffffff810fa3ce>] [<ffffffff810fa3ce>] native_queued_spin_lock_slowpath+0x1be/0x1e0
RSP: 0018:ffff883f685cbbc0 EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffffffffff10 RCX: 0000000000000001
RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff887f5e3b1480
RBP: ffff883f685cbbc0 R08: 0000000000000101 R09: 0000000003cc3a3c
R10: ffff883c7744b980 R11: 0000000000000000 R12: 0000000000000001
R13: 0000000000000101 R14: 0000000000000001 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff887f6c280000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000342147000 CR3: 00000000019f2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff883f685cbbd0 ffffffff8169e5ff ffff883f685cbbe0 ffffffff816abb50
ffff883f685cbc58 ffffffffc07b17eb ffff887f5e3b1480 0000000000000080
0000000000000000 ffffffff8121e724 0000000000000000 ffff883f685cbc18
Call Trace:
[<ffffffff8169e5ff>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff816abb50>] _raw_spin_lock+0x20/0x30
[<ffffffffc07b17eb>] __ext4_es_shrink+0x5b/0x2a0 [ext4]
[<ffffffff8121e724>] ? prune_icache_sb+0x174/0x340
[<ffffffffc07b1ae2>] ext4_es_shrink+0xb2/0x130 [ext4]
[<ffffffff81195413>] shrink_slab+0x163/0x330
[<ffffffff811f7537>] ? vmpressure+0x87/0x90
[<ffffffff81199081>] balance_pgdat+0x4b1/0x5e0
[<ffffffff81199323>] kswapd+0x173/0x440
[<ffffffff810b1910>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811991b0>] ? balance_pgdat+0x5e0/0x5e0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 37 81 fe 00 01 00 00 74 f4 e9 95 fe ff ff 0f 1f 80 00 00 00 00 83 fa 01 75 11 0f 1f 00 e9 69 fe ff ff 0f 1f 00 85 c0 74 0c f3 90 <8b> 07 0f b6 c0 83 f8 03 75 f0 b8 01 00 00 00 66 89 07 5d c3 f3
Complete report #230376
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
native_queued_spin_lock_slowpath
|
vmlinux |
0xffffffff9811228e
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x1ce
|
|
2
|
queued_spin_lock_slowpath
|
vmlinux |
0xffffffff9875bfcb
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xb
|
|
3
|
_raw_spin_lock
|
vmlinux |
0xffffffff9876a480
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x20
|
|
4
|
__ext4_es_shrink
|
ext4 |
0xffffffffc172aadb
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x5b
|
|
5
|
ext4_es_shrink
|
ext4 |
0xffffffffc172add4
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xb4
|
|
6
|
shrink_slab
|
vmlinux |
0xffffffff981c9da5
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x175
|
|
7
|
|
vmlinux |
0xffffffff98237527
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x87
|
|
8
|
balance_pgdat
|
vmlinux |
0xffffffff981cdba1
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x4b1
|
|
9
|
kswapd
|
vmlinux |
0xffffffff981cde43
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x173
|
|
10
|
|
vmlinux |
0xffffffff980c2d00
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x30
|
|
11
|
|
vmlinux |
0xffffffff981cdcd0
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x5e0
|
|
12
|
kthread
|
vmlinux |
0xffffffff980c1c31
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xd1
|
|
13
|
|
vmlinux |
0xffffffff980c1b60
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x40
|
|
14
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff98774c1d
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x7
|
|
15
|
|
vmlinux |
0xffffffff980c1b60
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kswapd0:306]
Modules linked in: binfmt_misc nvidia_uvm(POE) veth ip_gre gre ip_tunnel xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat overlay(T) dm_mirror dm_region_hash dm_log dm_mod iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ipmi_ssif mei_me lpc_ich i2c_i801 mei ipmi_si acpi_power_meter acpi_pad nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack libcrc32c ip_tables ext4 mbcache jbd2 nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) ast i2c_algo_bit drm_kms_helper crct10dif_pclmul syscopyarea crct10dif_common ttm sysfillrect
crc32c_intel sysimgblt fb_sys_fops i40e drm ahci nvme libahci nvme_core ipmi_devintf ipmi_msghandler ptp pps_core libata drm_panel_orientation_quirks wmi nfit libnvdimm br_netfilter bridge stp llc
CPU: 0 PID: 306 Comm: kswapd0 Kdump: loaded Tainted: P OE ------------ T 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: Inspur SA5212M5/YZMB-00882-104, BIOS 4.0.2 07/05/2018
task: ffff8a22f88030c0 ti: ffff8a22f8508000 task.ti: ffff8a22f8508000
RIP: 0010:[<ffffffff9811228e>] [<ffffffff9811228e>] native_queued_spin_lock_slowpath+0x1ce/0x200
RSP: 0018:ffff8a22f850bbc0 EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffff8a22f8803128 RCX: 0000000000000001
RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff8a22f871a480
RBP: ffff8a22f850bbc0 R08: 0000000000000101 R09: 000000000102f890
R10: 0000000000000003 R11: 0000000000000000 R12: ffff8a22deb8c400
R13: ffff8a226ccfc5a0 R14: ffff8a22fc21ab80 R15: 0000000046001992
FS: 0000000000000000(0000) GS:ffff8a22fc200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c4201ef010 CR3: 0000001003e10000 CR4: 00000000007607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffff9875bfcb>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9876a480>] _raw_spin_lock+0x20/0x30
[<ffffffffc172aadb>] __ext4_es_shrink+0x5b/0x2a0 [ext4]
[<ffffffffc172add4>] ext4_es_shrink+0xb4/0x130 [ext4]
[<ffffffff981c9da5>] shrink_slab+0x175/0x340
[<ffffffff98237527>] ? vmpressure+0x87/0x90
[<ffffffff981cdba1>] balance_pgdat+0x4b1/0x5e0
[<ffffffff981cde43>] kswapd+0x173/0x440
[<ffffffff980c2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff981cdcd0>] ? balance_pgdat+0x5e0/0x5e0
[<ffffffff980c1c31>] kthread+0xd1/0xe0
[<ffffffff980c1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff98774c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff980c1b60>] ? insert_kthread_work+0x40/0x40
Code: 37 81 fe 00 01 00 00 74 f4 e9 93 fe ff ff 0f 1f 80 00 00 00 00 83 fa 01 75 11 0f 1f 00 e9 68 fe ff ff 0f 1f 00 85 c0 74 0c f3 90 <8b> 07 0f b6 c0 83 f8 03 75 f0 b8 01 00 00 00 66 89 07 5d c3 66
Complete report #167215
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
native_queued_spin_lock_slowpath
|
vmlinux |
0xffffffffa2112222
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x122
|
|
2
|
queued_spin_lock_slowpath
|
vmlinux |
0xffffffffa275bfcb
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xb
|
|
3
|
_raw_spin_lock
|
vmlinux |
0xffffffffa276a480
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x20
|
|
4
|
__ext4_es_shrink
|
ext4 |
0xffffffffc0396adb
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x5b
|
|
5
|
|
vmlinux |
0xffffffffa225fefc
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x17c
|
|
6
|
ext4_es_shrink
|
ext4 |
0xffffffffc0396dd4
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xb4
|
|
7
|
shrink_slab
|
vmlinux |
0xffffffffa21c9de5
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x175
|
|
8
|
|
vmlinux |
0xffffffffa22376c1
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x61
|
|
9
|
balance_pgdat
|
vmlinux |
0xffffffffa21cdbe1
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x4b1
|
|
10
|
kswapd
|
vmlinux |
0xffffffffa21cde83
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x173
|
|
11
|
|
vmlinux |
0xffffffffa20c2d40
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x30
|
|
12
|
|
vmlinux |
0xffffffffa21cdd10
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x5e0
|
|
13
|
kthread
|
vmlinux |
0xffffffffa20c1c71
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xd1
|
|
14
|
|
vmlinux |
0xffffffffa20c1ba0
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x40
|
|
15
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffa2774c1d
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x7
|
|
16
|
|
vmlinux |
0xffffffffa20c1ba0
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kswapd0:270]
Modules linked in: veth nvidia_uvm(OE) nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel sr_mod iTCO_wdt iTCO_vendor_support snd_hda_codec_hdmi
kvm cdrom irqbypass joydev snd_hda_intel snd_hda_codec crc32_pclmul ghash_clmulni_intel snd_hda_core snd_hwdep aesni_intel snd_seq lrw gf128mul snd_seq_device glue_helper snd_pcm snd_timer snd soundcore ablk_helper cryptd i2c_i801 mei_me mei sg pcspkr lpc_ich ipmi_si ipmi_devintf acpi_pad ipmi_msghandler acpi_power_meter ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic uas usb_storage nouveau crct10dif_pclmul crct10dif_common crc32c_intel video mxm_wmi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw ahci igb libahci libata ptp pps_core drm_panel_orientation_quirks dca i2c_algo_bit wmi nfit libnvdimm
CPU: 0 PID: 270 Comm: kswapd0 Kdump: loaded Tainted: P OE ------------ T 3.10.0-957.5.1.el7.x86_64 #1
Hardware name: ASUSTeK COMPUTER INC. ESC4000 G4/Z11PG-D16 Series, BIOS 0601 03/01/2018
task: ffff8c5c1bd58000 ti: ffff8c5c1bd54000 task.ti: ffff8c5c1bd54000
RIP: 0010:[<ffffffffa2112222>] [<ffffffffa2112222>] native_queued_spin_lock_slowpath+0x122/0x200
RSP: 0000:ffff8c5c1bd57bc0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8c5c07ee41d8 RCX: 0000000000010000
RDX: ffff8c741be5b780 RSI: 0000000000690101 RDI: ffff8c7404fdbc80
RBP: ffff8c5c1bd57bc0 R08: ffff8c5c1da1b780 R09: 0000000000000000
R10: 000000000000b4c4 R11: 0000000000000001 R12: 0000000000000019
R13: ffff993984190f30 R14: ffff8c5bedc00080 R15: 0000000000a0df34
FS: 0000000000000000(0000) GS:ffff8c5c1da00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001cb10f0 CR3: 0000002014010000 CR4: 00000000007607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffa275bfcb>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffa276a480>] _raw_spin_lock+0x20/0x30
[<ffffffffc0396adb>] __ext4_es_shrink+0x5b/0x2a0 [ext4]
[<ffffffffa225fefc>] ? prune_icache_sb+0x17c/0x330
[<ffffffffc0396dd4>] ext4_es_shrink+0xb4/0x130 [ext4]
[<ffffffffa21c9de5>] shrink_slab+0x175/0x340
[<ffffffffa22376c1>] ? vmpressure+0x61/0x90
[<ffffffffa21cdbe1>] balance_pgdat+0x4b1/0x5e0
[<ffffffffa21cde83>] kswapd+0x173/0x440
[<ffffffffa20c2d40>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffa21cdd10>] ? balance_pgdat+0x5e0/0x5e0
[<ffffffffa20c1c71>] kthread+0xd1/0xe0
[<ffffffffa20c1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa2774c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa20c1ba0>] ? insert_kthread_work+0x40/0x40
Code: 13 48 c1 ea 0d 48 98 83 e2 30 48 81 c2 80 b7 01 00 48 03 14 c5 60 b9 d4 a2 4c 89 02 41 8b 40 08 85 c0 75 0f 0f 1f 44 00 00 f3 90 <41> 8b 40 08 85 c0 74 f6 4d 8b 08 4d 85 c9 74 04 41 0f 18 09 8b
Complete report #15290
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
native_queued_spin_lock_slowpath
|
vmlinux |
0xffffffffb3118080
Build id: 3.10.0-1127.el7.x86_64
|
0x1d0
|
|
2
|
queued_spin_lock_slowpath
|
vmlinux |
0xffffffffb377a004
Build id: 3.10.0-1127.el7.x86_64
|
0xb
|
|
3
|
_raw_spin_lock
|
vmlinux |
0xffffffffb37886b0
Build id: 3.10.0-1127.el7.x86_64
|
0x20
|
|
4
|
__ext4_es_shrink
|
ext4 |
0xffffffffc10107cb
Build id: 3.10.0-1127.el7.x86_64
|
0x5b
|
|
5
|
|
vmlinux |
0xffffffffb326c7dc
Build id: 3.10.0-1127.el7.x86_64
|
0x17c
|
|
6
|
ext4_es_shrink
|
ext4 |
0xffffffffc1010ac4
Build id: 3.10.0-1127.el7.x86_64
|
0xb4
|
|
7
|
shrink_slab
|
vmlinux |
0xffffffffb31d1c85
Build id: 3.10.0-1127.el7.x86_64
|
0x175
|
|
8
|
|
vmlinux |
0xffffffffb32427e7
Build id: 3.10.0-1127.el7.x86_64
|
0x87
|
|
9
|
balance_pgdat
|
vmlinux |
0xffffffffb31d59b8
Build id: 3.10.0-1127.el7.x86_64
|
0x3a8
|
|
10
|
kswapd
|
vmlinux |
0xffffffffb31d5d63
Build id: 3.10.0-1127.el7.x86_64
|
0x173
|
|
11
|
|
vmlinux |
0xffffffffb30c7780
Build id: 3.10.0-1127.el7.x86_64
|
0x30
|
|
12
|
|
vmlinux |
0xffffffffb31d5bf0
Build id: 3.10.0-1127.el7.x86_64
|
0x5e0
|
|
13
|
kthread
|
vmlinux |
0xffffffffb30c6691
Build id: 3.10.0-1127.el7.x86_64
|
0xd1
|
|
14
|
|
vmlinux |
0xffffffffb30c65c0
Build id: 3.10.0-1127.el7.x86_64
|
0x40
|
|
15
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffb3792d37
Build id: 3.10.0-1127.el7.x86_64
|
0x21
|
|
16
|
|
vmlinux |
0xffffffffb30c65c0
Build id: 3.10.0-1127.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kswapd0:469]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache binfmt_misc ext4 mbcache jbd2 iTCO_wdt mxm_wmi iTCO_vendor_support dcdbas sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg ipmi_si ipmi_devintf ipmi_msghandler mei_me mei lpc_ich acpi_power_meter wmi nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci libata crct10dif_pclmul crct10dif_common crc32c_intel tg3 megaraid_sas drm_panel_orientation_quirks ptp pps_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 469 Comm: kswapd0 Kdump: loaded Tainted: G L ------------ 3.10.0-1127.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R730xd/072T6D, BIOS 2.1.7 06/16/2016
task: ffff8c67fe3e5230 ti: ffff8c07fc504000 task.ti: ffff8c07fc504000
RIP: 0010:[<ffffffffb3118080>] [<ffffffffb3118080>] native_queued_spin_lock_slowpath+0x1d0/0x200
RSP: 0018:ffff8c07fc507bc0 EFLAGS: 00000202
RAX: 0000000001810101 RBX: ffff8c07fc507b50 RCX: 0000000000000001
RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff8c43bf4a0480
RBP: ffff8c07fc507bc0 R08: 0000000000000101 R09: 0000000003c5fb04
R10: 0000000000000001 R11: ffff8c07e3ff91a0 R12: ffff8c277ffb8770
R13: ffff8c277ffb86c0 R14: ffffffffb3c7a240 R15: ffffb1a6b30bde20
FS: 0000000000000000(0000) GS:ffff8c07ff200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055d43eb2fc5c CR3: 0000009abf210000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb377a004>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffb37886b0>] _raw_spin_lock+0x20/0x30
[<ffffffffc10107cb>] __ext4_es_shrink+0x5b/0x2a0 [ext4]
[<ffffffffb326c7dc>] ? prune_icache_sb+0x17c/0x330
[<ffffffffc1010ac4>] ext4_es_shrink+0xb4/0x130 [ext4]
[<ffffffffb31d1c85>] shrink_slab+0x175/0x340
[<ffffffffb32427e7>] ? vmpressure+0x87/0x90
[<ffffffffb31d59b8>] balance_pgdat+0x3a8/0x5e0
[<ffffffffb31d5d63>] kswapd+0x173/0x430
[<ffffffffb30c7780>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffb31d5bf0>] ? balance_pgdat+0x5e0/0x5e0
[<ffffffffb30c6691>] kthread+0xd1/0xe0
[<ffffffffb30c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb3792d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb30c65c0>] ? insert_kthread_work+0x40/0x40
Code: fe 00 01 00 00 74 f4 e9 93 fe ff ff 0f 1f 80 00 00 00 00 83 fa 01 75 11 0f 1f 00 e9 68 fe ff ff 0f 1f 00 85 c0 74 0c f3 90 8b 07 <0f> b6 c0 83 f8 03 75 f0 b8 01 00 00 00 66 89 07 5d c3 66 0f 1f
Complete report #7542
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
native_queued_spin_lock_slowpath
|
vmlinux |
0xffffffffa190ba92
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x122
|
|
2
|
queued_spin_lock_slowpath
|
vmlinux |
0xffffffffa1f0dc5f
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xb
|
|
3
|
_raw_spin_lock
|
vmlinux |
0xffffffffa1f1b750
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x20
|
|
4
|
__ext4_es_shrink
|
ext4 |
0xffffffffc0a363ab
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x5b
|
|
5
|
ext4_es_shrink
|
ext4 |
0xffffffffc0a366a4
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xb4
|
|
6
|
shrink_slab
|
vmlinux |
0xffffffffa19aa065
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x175
|
|
7
|
|
vmlinux |
0xffffffffa1a157b7
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x87
|
|
8
|
balance_pgdat
|
vmlinux |
0xffffffffa19add21
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x4b1
|
|
9
|
kswapd
|
vmlinux |
0xffffffffa19adfc3
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x173
|
|
10
|
|
vmlinux |
0xffffffffa18bef10
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x30
|
|
11
|
|
vmlinux |
0xffffffffa19ade50
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x5e0
|
|
12
|
kthread
|
vmlinux |
0xffffffffa18bdf21
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xd1
|
|
13
|
|
vmlinux |
0xffffffffa18bde50
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x40
|
|
14
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffa1f255dd
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x7
|
|
15
|
|
vmlinux |
0xffffffffa18bde50
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kswapd0:270]
Modules linked in: fuse nfsd nfsv3 nfs_acl nfnetlink_queue nfnetlink_log xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc vfat fat ext4 mbcache jbd2 iTCO_wdt iTCO_vendor_support dcdbas skx_edac intel_powerclamp coretemp intel_rapl
iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif i2c_i801 pcspkr ipmi_si nfit sg ipmi_devintf ipmi_msghandler lpc_ich mei_me libnvdimm mei shpchp acpi_power_meter acpi_pad ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul ttm crct10dif_common crc32c_intel drm ahci i40e libahci libata megaraid_sas ptp i2c_core pps_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 270 Comm: kswapd0 Kdump: loaded Tainted: G L ------------ 3.10.0-862.14.4.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R640/008R9M, BIOS 1.2.71 12/04/2017
task: ffff98533a404f10 ti: ffff97f33b3d4000 task.ti: ffff97f33b3d4000
RIP: 0010:[<ffffffffa190ba92>] [<ffffffffa190ba92>] native_queued_spin_lock_slowpath+0x122/0x200
RSP: 0018:ffff97f33b3d7bc0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff97fb62a4e8d0 RCX: 0000000000110000
RDX: ffff98533c299740 RSI: 0000000001290001 RDI: ffff97f4ecf64c80
RBP: ffff97f33b3d7bc0 R08: ffff97f33da59740 R09: 0000000000000000
R10: 00000000ee8ca801 R11: ffffdbb549ba3280 R12: ffff985327e88000
R13: ffff9796ee8ca820 R14: ffff979b76241640 R15: 00000000882bc77f
FS: 0000000000000000(0000) GS:ffff97f33da40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000271a3a0 CR3: 0000001152df8000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffa1f0dc5f>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffa1f1b750>] _raw_spin_lock+0x20/0x30
[<ffffffffc0a363ab>] __ext4_es_shrink+0x5b/0x2a0 [ext4]
[<ffffffffc0a366a4>] ext4_es_shrink+0xb4/0x130 [ext4]
[<ffffffffa19aa065>] shrink_slab+0x175/0x340
[<ffffffffa1a157b7>] ? vmpressure+0x87/0x90
[<ffffffffa19add21>] balance_pgdat+0x4b1/0x5e0
[<ffffffffa19adfc3>] kswapd+0x173/0x440
[<ffffffffa18bef10>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffa19ade50>] ? balance_pgdat+0x5e0/0x5e0
[<ffffffffa18bdf21>] kthread+0xd1/0xe0
[<ffffffffa18bde50>] ? insert_kthread_work+0x40/0x40
[<ffffffffa1f255dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa18bde50>] ? insert_kthread_work+0x40/0x40
Code: 13 48 c1 ea 0d 48 98 83 e2 30 48 81 c2 40 97 01 00 48 03 14 c5 a0 54 53 a2 4c 89 02 41 8b 40 08 85 c0 75 0f 0f 1f 44 00 00 f3 90 <41> 8b 40 08 85 c0 74 f6 4d 8b 08 4d 85 c9 74 04 41 0f 18 09 8b