Info

Permalink
Function
e1000_alloc_rx_buffers
First occurrence
2020-09-26
Last occurrence
2026-10-04
State
NEW
Components
kernel
Quality
-27

Statistics

Operating system Count
CentOS 7 14
Architecture Count
x86_64 14
Related packages Count
kernel 14
 0:3.10.0-514.el7 10
 0:3.10.0-514.16.1.el7 3
 0:3.10.0-327.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #166553
Frame # Function Binary Source or offset Line
1
e1000_alloc_rx_buffers
e1000 0xffffffffa001355e
Build id: 3.10.0-327.el7.x86_64 		0x24e
2
e1000_clean_rx_irq
e1000 0xffffffffa001316d
Build id: 3.10.0-327.el7.x86_64 		0x3dd
3
e1000_clean
e1000 0xffffffffa0015a75
Build id: 3.10.0-327.el7.x86_64 		0x265
4
? update_curr
vmlinux 0xffffffff810c14bc
Build id: 3.10.0-327.el7.x86_64 		0xcc
5
? account_entity_dequeue
vmlinux 0xffffffff810be2ee
Build id: 3.10.0-327.el7.x86_64 		0xae
6
net_rx_action
vmlinux 0xffffffff8152afa2
Build id: 3.10.0-327.el7.x86_64 		0x152
7
__do_softirq
vmlinux 0xffffffff81084b0f
Build id: 3.10.0-327.el7.x86_64 		0xef
8
run_ksoftirqd
vmlinux 0xffffffff81084cd0
Build id: 3.10.0-327.el7.x86_64 		0x30
9
smpboot_thread_fn
vmlinux 0xffffffff810ae52f
Build id: 3.10.0-327.el7.x86_64 		0xff
10
? schedule
vmlinux 0xffffffff8163a909
Build id: 3.10.0-327.el7.x86_64 		0x29
11
? lg_double_unlock
vmlinux 0xffffffff810ae430
Build id: 3.10.0-327.el7.x86_64 		0x90
12
kthread
vmlinux 0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64 		0xcf
13
? kthread_create_on_node
vmlinux 0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64 		0x140
14
ret_from_fork
vmlinux 0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64 		0x58
15
? kthread_create_on_node
vmlinux 0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64 		0x140 
BUG: soft lockup - CPU#0 stuck for 23s! [ksoftirqd/0:3]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter fuse vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event coretemp snd_ens1371 snd_rawmidi snd_ac97_codec crc32_pclmul ghash_clmulni_intel ac97_bus aesni_intel lrw snd_seq snd_seq_device gf128mul snd_pcm snd_timer glue_helper ablk_helper snd soundcore cryptd pcspkr sg ppdev vmw_balloon parport_pc parport i2c_piix4 shpchp vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi crct10dif_pclmul
crct10dif_common vmwgfx drm_kms_helper ttm crc32c_intel drm serio_raw mptspi scsi_transport_spi mptscsih mptbase ata_piix libata e1000 i2c_core
CPU: 0 PID: 3 Comm: ksoftirqd/0 Tainted: G             L ------------   3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8802331b1700 ti: ffff880233234000 task.ti: ffff880233234000
RIP: 0010:[<ffffffffa001355e>]  [<ffffffffa001355e>] e1000_alloc_rx_buffers+0x24e/0x450 [e1000]
RSP: 0018:ffff880233237bd0  EFLAGS: 00000286
RAX: ffffc90008d02818 RBX: 0000000000000202 RCX: 00000000bd75e800
RDX: 00000000000000c6 RSI: 00000000bd75e800 RDI: ffff88023238a098
RBP: ffff880233237c20 R08: 0000000000000202 R09: 0000000000000000
R10: 000060fdc8801840 R11: 0000000000000001 R12: 0000000200000780
R13: ffffffff81313f7e R14: ffff880233237b98 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff880236600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f08b371c978 CR3: 0000000230daf000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880233237c00 ffff880230f8c000 00000000000005f2 ffff880230f8c8c0
ffff88023238a000 0000000000000000 00000000000000c8 ffff880095f3d200
ffff8800b9cc5c80 ffff8800b9cc5c70 ffff880233237ce0 ffffffffa001316d
Call Trace:
[<ffffffffa001316d>] e1000_clean_rx_irq+0x3dd/0x580 [e1000]
[<ffffffffa0015a75>] e1000_clean+0x265/0x8e0 [e1000]
[<ffffffff810c14bc>] ? update_curr+0xcc/0x150
[<ffffffff810be2ee>] ? account_entity_dequeue+0xae/0xd0
[<ffffffff8152afa2>] net_rx_action+0x152/0x240
[<ffffffff81084b0f>] __do_softirq+0xef/0x280
[<ffffffff81084cd0>] run_ksoftirqd+0x30/0x50
[<ffffffff810ae52f>] smpboot_thread_fn+0xff/0x1a0
[<ffffffff8163a909>] ? schedule+0x29/0x70
[<ffffffff810ae430>] ? lg_double_unlock+0x90/0x90
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: 67 18 74 27 45 85 e4 45 89 67 18 41 8d 54 24 ff 0f 84 fc 01 00 00 0f ae f8 48 8b 5d c8 41 0f b7 47 36 48 03 83 d0 03 00 00 89 10 <48> 83 c4 28 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 48 85 c0
Complete report #100435
Frame # Function Binary Source or offset Line
1
e1000_alloc_rx_buffers
e1000 0xffffffffa00e48ae
Build id: 3.10.0-514.el7.x86_64 		0x36e
2
? netif_receive_skb_internal
vmlinux 0xffffffff8156f8a0
Build id: 3.10.0-514.el7.x86_64 		0x40
3
e1000_clean_rx_irq
e1000 0xffffffffa00e4426
Build id: 3.10.0-514.el7.x86_64 		0x3d6
4
e1000_clean
e1000 0xffffffffa00e4d31
Build id: 3.10.0-514.el7.x86_64 		0x281
5
net_rx_action
vmlinux 0xffffffff815700b0
Build id: 3.10.0-514.el7.x86_64 		0x170
6
__do_softirq
vmlinux 0xffffffff8108f0ff
Build id: 3.10.0-514.el7.x86_64 		0xef
7
run_ksoftirqd
vmlinux 0xffffffff8108f2c8
Build id: 3.10.0-514.el7.x86_64 		0x38
8
smpboot_thread_fn
vmlinux 0xffffffff810b90af
Build id: 3.10.0-514.el7.x86_64 		0x12f
9
? lg_double_unlock
vmlinux 0xffffffff810b8f80
Build id: 3.10.0-514.el7.x86_64 		0x90
10
kthread
vmlinux 0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64 		0xcf
11
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140
12
ret_from_fork
vmlinux 0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64 		0x58
13
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 29s! [ksoftirqd/4:28]
Modules linked in: fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun nf_conntrack_ftp ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq pcspkr snd_seq_device
snd_pcm snd_timer sg snd vmw_vmci soundcore i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi crct10dif_pclmul vmwgfx crct10dif_common crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi serio_raw ata_piix scsi_transport_spi drm mptscsih e1000 libata mptbase i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 28 Comm: ksoftirqd/4 Tainted: G             L ------------   3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/20/2014
task: ffff880181f19f60 ti: ffff880181f24000 task.ti: ffff880181f24000
RIP: 0010:[<ffffffffa00e48ae>]  [<ffffffffa00e48ae>] e1000_alloc_rx_buffers+0x36e/0x570 [e1000]
RSP: 0018:ffff880181f27bd8  EFLAGS: 00000286
RAX: ffffc90001902818 RBX: ffff880181f27bb8 RCX: 00000000000005f2
RDX: 00000000000000ed RSI: ffff880035d5b840 RDI: ffff88042d4868c0
RBP: ffff880181f27c30 R08: 0000000000019b00 R09: ffffffff8155bf77
R10: ffff88042d719b00 R11: ffffea0010963300 R12: ffffffff81aeae78
R13: ffff8804258cc600 R14: 00000000d6b312e4 R15: ffffffff810d1568
FS:  0000000000000000(0000) GS:ffff88042d700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffc10500160 CR3: 0000000428a8b000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
000005f281f27c08 ffffffff8156f8a0 00000000000005f1 ffff88042d486b70
ffff88042e02a000 ffff88042e02a098 ffff880035d5b840 ffff88042d4868c0
ffff88003521eef0 ffff8804258cc600 ffff88003521eef0 ffff880181f27cd8
Call Trace:
[<ffffffff8156f8a0>] ? netif_receive_skb_internal+0x40/0xc0
[<ffffffffa00e4426>] e1000_clean_rx_irq+0x3d6/0x4f0 [e1000]
[<ffffffffa00e4d31>] e1000_clean+0x281/0x8f0 [e1000]
[<ffffffff815700b0>] net_rx_action+0x170/0x380
[<ffffffff8108f0ff>] __do_softirq+0xef/0x280
[<ffffffff8108f2c8>] run_ksoftirqd+0x38/0x50
[<ffffffff810b90af>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f80>] ? lg_double_unlock+0x90/0x90
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 00 01 0f 1f 00 41 3b 5d 18 74 20 85 db 41 89 5d 18 8d 53 ff 0f 84 e0 01 00 00 0f ae f8 41 0f b7 45 36 49 03 87 d0 03 00 00 89 10 <48> 83 c4 30 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 48 8b 45
Complete report #97704
Frame # Function Binary Source or offset Line
1
e1000_alloc_rx_buffers
e1000 0xffffffffa00cc8ae
Build id: 3.10.0-514.el7.x86_64 		0x36e
2
? netif_receive_skb_internal
vmlinux 0xffffffff8156f8a0
Build id: 3.10.0-514.el7.x86_64 		0x40
3
e1000_clean_rx_irq
e1000 0xffffffffa00cc4a5
Build id: 3.10.0-514.el7.x86_64 		0x455
4
e1000_clean
e1000 0xffffffffa00ccd31
Build id: 3.10.0-514.el7.x86_64 		0x281
5
? __queue_work
vmlinux 0xffffffff810a5d50
Build id: 3.10.0-514.el7.x86_64 		0x140
6
net_rx_action
vmlinux 0xffffffff815700b0
Build id: 3.10.0-514.el7.x86_64 		0x170
7
__do_softirq
vmlinux 0xffffffff8108f0ff
Build id: 3.10.0-514.el7.x86_64 		0xef
8
run_ksoftirqd
vmlinux 0xffffffff8108f2c8
Build id: 3.10.0-514.el7.x86_64 		0x38
9
smpboot_thread_fn
vmlinux 0xffffffff810b90af
Build id: 3.10.0-514.el7.x86_64 		0x12f
10
? lg_double_unlock
vmlinux 0xffffffff810b8f80
Build id: 3.10.0-514.el7.x86_64 		0x90
11
kthread
vmlinux 0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64 		0xcf
12
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140
13
ret_from_fork
vmlinux 0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64 		0x58
14
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [ksoftirqd/4:28]
Modules linked in: tcp_lp binfmt_misc veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371
snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm vmw_balloon ppdev snd_timer pcspkr snd shpchp soundcore vmw_vmci sg i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi serio_raw drm scsi_transport_spi ata_piix mptscsih e1000 libata mptbase i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 28 Comm: ksoftirqd/4 Tainted: G             L ------------ T 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/20/2014
task: ffff880179761f60 ti: ffff880179770000 task.ti: ffff880179770000
RIP: 0010:[<ffffffffa00cc8ae>]  [<ffffffffa00cc8ae>] e1000_alloc_rx_buffers+0x36e/0x570 [e1000]
RSP: 0018:ffff880179773bd8  EFLAGS: 00000286
RAX: ffffc900060c2818 RBX: ffff880179773bb8 RCX: 00000000000005f2
RDX: 0000000000000040 RSI: ffff880ff73d8180 RDI: ffff880ffcfbe8c0
RBP: ffff880179773c30 R08: 0000000000019b00 R09: ffffffff8155bf77
R10: ffff880ffd719b00 R11: ffffea0002222900 R12: ffffffff81aeae78
R13: ffff8800888a5300 R14: 00000000f2d0ad2f R15: ffffffff810cde74
FS:  0000000000000000(0000) GS:ffff880ffd700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000007801c8 CR3: 0000000fd2f2a000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
000005f279773c08 ffffffff8156f8a0 00000000000005f1 ffff880ffcfbeb70
ffff880ffd82c000 ffff880ffd82c098 0000000000000000 ffff880ffcfbe8c0
ffff8800350b5410 ffff8800888a5300 ffff8800350b5420 ffff880179773cd8
Call Trace:
[<ffffffff8156f8a0>] ? netif_receive_skb_internal+0x40/0xc0
[<ffffffffa00cc4a5>] e1000_clean_rx_irq+0x455/0x4f0 [e1000]
[<ffffffffa00ccd31>] e1000_clean+0x281/0x8f0 [e1000]
[<ffffffff810a5d50>] ? __queue_work+0x140/0x320
[<ffffffff815700b0>] net_rx_action+0x170/0x380
[<ffffffff8108f0ff>] __do_softirq+0xef/0x280
[<ffffffff8108f2c8>] run_ksoftirqd+0x38/0x50
[<ffffffff810b90af>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f80>] ? lg_double_unlock+0x90/0x90
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 00 01 0f 1f 00 41 3b 5d 18 74 20 85 db 41 89 5d 18 8d 53 ff 0f 84 e0 01 00 00 0f ae f8 41 0f b7 45 36 49 03 87 d0 03 00 00 89 10 <48> 83 c4 30 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 48 8b 45
Complete report #87007
Frame # Function Binary Source or offset Line
1
e1000_alloc_rx_buffers
e1000 0xffffffffa012d8ae
Build id: 3.10.0-514.el7.x86_64 		0x36e
2
e1000_clean_rx_irq
e1000 0xffffffffa012d4a5
Build id: 3.10.0-514.el7.x86_64 		0x455
3
e1000_clean
e1000 0xffffffffa012dd31
Build id: 3.10.0-514.el7.x86_64 		0x281
4
net_rx_action
vmlinux 0xffffffff815700b0
Build id: 3.10.0-514.el7.x86_64 		0x170
5
__do_softirq
vmlinux 0xffffffff8108f0ff
Build id: 3.10.0-514.el7.x86_64 		0xef
6
run_ksoftirqd
vmlinux 0xffffffff8108f2c8
Build id: 3.10.0-514.el7.x86_64 		0x38
7
smpboot_thread_fn
vmlinux 0xffffffff810b90af
Build id: 3.10.0-514.el7.x86_64 		0x12f
8
? lg_double_unlock
vmlinux 0xffffffff810b8f80
Build id: 3.10.0-514.el7.x86_64 		0x90
9
kthread
vmlinux 0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64 		0xcf
10
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140
11
ret_from_fork
vmlinux 0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64 		0x58
12
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ksoftirqd/0:3]
Modules linked in: veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_nat_ipv4 nf_nat br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock nf_conntrack_ipv6 nf_defrag_ipv6 xt_conntrack ip6table_filter ip6_tables snd_seq_midi snd_seq_midi_event intel_powerclamp coretemp iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi vmw_balloon snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm pcspkr snd_timer snd soundcore sg i2c_piix4 vmw_vmci shpchp nfit libnvdimm parport_pc parport nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul
crct10dif_common crc32c_intel vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm drm ata_piix mptspi scsi_transport_spi mptscsih e1000 mptbase i2c_core libata fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3 Comm: ksoftirqd/0 Tainted: G             L ------------ T 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff880149731f60 ti: ffff880149038000 task.ti: ffff880149038000
RIP: 0010:[<ffffffffa012d8ae>]  [<ffffffffa012d8ae>] e1000_alloc_rx_buffers+0x36e/0x570 [e1000]
RSP: 0000:ffff88014903bbd8  EFLAGS: 00000286
RAX: ffffc90000d02818 RBX: 0000000136458fc0 RCX: ffff880148a45000
RDX: 000000000000004d RSI: 00000000bcc02800 RDI: 00000000000005f2
RBP: ffff88014903bc30 R08: 00000000000005f2 R09: 0000000000000000
R10: ffff880136458fc0 R11: ffffea00010ad900 R12: ffffffff8133abbe
R13: ffff88014903bba0 R14: 0000000000000001 R15: 0000000000000202
FS:  0000000000000000(0000) GS:ffff88014b800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000c0007e5b38 CR3: 000000006476c000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
000005f24903bc08 ffff8801b6458fc0 00000000000005f1 ffff880136458fc0
ffff880148a45000 ffff880148a45098 0000000000000000 ffff88014703e8c0
ffff880038c554e0 ffff880042b66100 ffff880038c554f0 ffff88014903bcd8
Call Trace:
[<ffffffffa012d4a5>] e1000_clean_rx_irq+0x455/0x4f0 [e1000]
[<ffffffffa012dd31>] e1000_clean+0x281/0x8f0 [e1000]
[<ffffffff815700b0>] net_rx_action+0x170/0x380
[<ffffffff8108f0ff>] __do_softirq+0xef/0x280
[<ffffffff8108f2c8>] run_ksoftirqd+0x38/0x50
[<ffffffff810b90af>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f80>] ? lg_double_unlock+0x90/0x90
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 00 01 0f 1f 00 41 3b 5d 18 74 20 85 db 41 89 5d 18 8d 53 ff 0f 84 e0 01 00 00 0f ae f8 41 0f b7 45 36 49 03 87 d0 03 00 00 89 10 <48> 83 c4 30 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 48 8b 45
Complete report #23242
Frame # Function Binary Source or offset Line
1
e1000_alloc_rx_buffers
e1000 0xffffffffa00d18ae
Build id: 3.10.0-514.el7.x86_64 		0x36e
2
? netif_receive_skb_internal
vmlinux 0xffffffff8156f8a0
Build id: 3.10.0-514.el7.x86_64 		0x40
3
e1000_clean_rx_irq
e1000 0xffffffffa00d14a5
Build id: 3.10.0-514.el7.x86_64 		0x455
4
e1000_clean
e1000 0xffffffffa00d1d31
Build id: 3.10.0-514.el7.x86_64 		0x281
5
? try_to_wake_up
vmlinux 0xffffffff810c4cf8
Build id: 3.10.0-514.el7.x86_64 		0x1c8
6
net_rx_action
vmlinux 0xffffffff815700b0
Build id: 3.10.0-514.el7.x86_64 		0x170
7
__do_softirq
vmlinux 0xffffffff8108f0ff
Build id: 3.10.0-514.el7.x86_64 		0xef
8
run_ksoftirqd
vmlinux 0xffffffff8108f2c8
Build id: 3.10.0-514.el7.x86_64 		0x38
9
smpboot_thread_fn
vmlinux 0xffffffff810b90af
Build id: 3.10.0-514.el7.x86_64 		0x12f
10
? lg_double_unlock
vmlinux 0xffffffff810b8f80
Build id: 3.10.0-514.el7.x86_64 		0x90
11
kthread
vmlinux 0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64 		0xcf
12
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140
13
ret_from_fork
vmlinux 0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64 		0x58
14
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 43s! [ksoftirqd/1:13]
Modules linked in: rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus vmw_balloon snd_seq snd_seq_device snd_pcm btusb btrtl btbcm btintel
pcspkr bluetooth snd_timer rfkill snd soundcore sg vmw_vmci i2c_piix4 shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic crct10dif_pclmul crct10dif_common crc32c_intel pata_acpi mptspi scsi_transport_spi mptscsih serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix e1000 i2c_core libata mptbase fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 13 Comm: ksoftirqd/1 Not tainted 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff88013707af10 ti: ffff880137090000 task.ti: ffff880137090000
RIP: 0010:[<ffffffffa00d18ae>]  [<ffffffffa00d18ae>] e1000_alloc_rx_buffers+0x36e/0x570 [e1000]
RSP: 0018:ffff880137093bd8  EFLAGS: 00000286
RAX: ffffc90000702818 RBX: ffff880137093bb8 RCX: 00000000000005f2
RDX: 0000000000000068 RSI: ffff880036183780 RDI: ffff880135a588c0
RBP: ffff880137093c30 R08: 0000000000019b00 R09: ffffffff8155bf77
R10: ffff880139659b00 R11: ffffea00027d2900 R12: ffffffff81aeae78
R13: ffff88009f4a6a00 R14: 0000000094dfcacf R15: ffff8800b1c6fd00
FS:  00007efe2e946a40(0000) GS:ffff880139640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5ea0770ba8 CR3: 00000000af45e000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
000005f237093c08 ffffffff8156f8a0 00000000000005f1 ffff880135a58b70
ffff880136f54000 ffff880136f54098 0000000000000000 ffff880135a588c0
ffff8800b5a93690 ffff88009f4a6a00 ffff8800b5a936a0 ffff880137093cd8
Call Trace:
[<ffffffff8156f8a0>] ? netif_receive_skb_internal+0x40/0xc0
[<ffffffffa00d14a5>] e1000_clean_rx_irq+0x455/0x4f0 [e1000]
[<ffffffffa00d1d31>] e1000_clean+0x281/0x8f0 [e1000]
[<ffffffff810c4cf8>] ? try_to_wake_up+0x1c8/0x330
[<ffffffff815700b0>] net_rx_action+0x170/0x380
[<ffffffff8108f0ff>] __do_softirq+0xef/0x280
[<ffffffff8108f2c8>] run_ksoftirqd+0x38/0x50
[<ffffffff810b90af>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f80>] ? lg_double_unlock+0x90/0x90
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 00 01 0f 1f 00 41 3b 5d 18 74 20 85 db 41 89 5d 18 8d 53 ff 0f 84 e0 01 00 00 0f ae f8 41 0f b7 45 36 49 03 87 d0 03 00 00 89 10 <48> 83 c4 30 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 48 8b 45
Complete report #16947
Frame # Function Binary Source or offset Line
1
e1000_alloc_rx_buffers
e1000 0xffffffffa00888ae
Build id: 3.10.0-514.el7.x86_64 		0x36e
2
e1000_clean_rx_irq
e1000 0xffffffffa00884a5
Build id: 3.10.0-514.el7.x86_64 		0x455
3
e1000_clean
e1000 0xffffffffa0088d31
Build id: 3.10.0-514.el7.x86_64 		0x281
4
? sched_clock
vmlinux 0xffffffff810336e9
Build id: 3.10.0-514.el7.x86_64 		0x9
5
net_rx_action
vmlinux 0xffffffff815700b0
Build id: 3.10.0-514.el7.x86_64 		0x170
6
__do_softirq
vmlinux 0xffffffff8108f0ff
Build id: 3.10.0-514.el7.x86_64 		0xef
7
run_ksoftirqd
vmlinux 0xffffffff8108f2c8
Build id: 3.10.0-514.el7.x86_64 		0x38
8
smpboot_thread_fn
vmlinux 0xffffffff810b90af
Build id: 3.10.0-514.el7.x86_64 		0x12f
9
? lg_double_unlock
vmlinux 0xffffffff810b8f80
Build id: 3.10.0-514.el7.x86_64 		0x90
10
kthread
vmlinux 0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64 		0xcf
11
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140
12
ret_from_fork
vmlinux 0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64 		0x58
13
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [ksoftirqd/2:18]
Modules linked in: fuse nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables devlink tun bridge stp llc ip_set nfnetlink bnep vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 intel_powerclamp snd_rawmidi snd_ac97_codec coretemp ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd iosf_mbi crc32_pclmul ghash_clmulni_intel
aesni_intel lrw gf128mul btusb btrtl btbcm glue_helper btintel ablk_helper bluetooth ppdev cryptd rfkill soundcore vmw_balloon sg pcspkr nfit libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ata_piix fb_sys_fops ttm serio_raw libata mptspi drm scsi_transport_spi mptscsih e1000 mptbase i2c_core fjes dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 2 PID: 18 Comm: ksoftirqd/2 Tainted: G             L ------------ T 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff88007a150000 ti: ffff88007a14c000 task.ti: ffff88007a14c000
RIP: 0010:[<ffffffffa00888ae>]  [<ffffffffa00888ae>] e1000_alloc_rx_buffers+0x36e/0x570 [e1000]
RSP: 0000:ffff88007a14fbd8  EFLAGS: 00000286
RAX: ffffc90000382818 RBX: 0000000045330000 RCX: ffff880079df5000
RDX: 0000000000000022 RSI: ffff880079df5098 RDI: 00000000000005f2
RBP: ffff88007a14fc30 R08: 00000000000005f2 R09: 0000000078a386b1
R10: ffff880078a380c0 R11: ffffea0000119100 R12: 0000000000000000
R13: 000000000000004e R14: ffff880078aceba8 R15: ffffffff815db92b
FS:  0000000000000000(0000) GS:ffff88007b680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc371154000 CR3: 000000004bb12000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
000005f224ea5a2b ffff8800f8a380c0 00000000000005f1 ffff880078a380c0
ffff880079df5000 ffff880079df5098 0000000000000000 ffff880078ace8c0
ffff880076a48230 ffff880004647300 ffff880076a48240 ffff88007a14fcd8
Call Trace:
[<ffffffffa00884a5>] e1000_clean_rx_irq+0x455/0x4f0 [e1000]
[<ffffffffa0088d31>] e1000_clean+0x281/0x8f0 [e1000]
[<ffffffff810336e9>] ? sched_clock+0x9/0x10
[<ffffffff815700b0>] net_rx_action+0x170/0x380
[<ffffffff8108f0ff>] __do_softirq+0xef/0x280
[<ffffffff8108f2c8>] run_ksoftirqd+0x38/0x50
[<ffffffff810b90af>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f80>] ? lg_double_unlock+0x90/0x90
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 00 01 0f 1f 00 41 3b 5d 18 74 20 85 db 41 89 5d 18 8d 53 ff 0f 84 e0 01 00 00 0f ae f8 41 0f b7 45 36 49 03 87 d0 03 00 00 89 10 <48> 83 c4 30 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 48 8b 45
Complete report #2628
Frame # Function Binary Source or offset Line
1
e1000_alloc_rx_buffers
e1000 0xffffffffa00cc8ae
Build id: 3.10.0-514.el7.x86_64 		0x36e
2
? netif_receive_skb_internal
vmlinux 0xffffffff8156f8a0
Build id: 3.10.0-514.el7.x86_64 		0x40
3
e1000_clean_rx_irq
e1000 0xffffffffa00cc4a5
Build id: 3.10.0-514.el7.x86_64 		0x455
4
e1000_clean
e1000 0xffffffffa00ccd31
Build id: 3.10.0-514.el7.x86_64 		0x281
5
? tcp_init_xmit_timers
vmlinux 0xffffffff815d16a0
Build id: 3.10.0-514.el7.x86_64 		0x30
6
net_rx_action
vmlinux 0xffffffff815700b0
Build id: 3.10.0-514.el7.x86_64 		0x170
7
__do_softirq
vmlinux 0xffffffff8108f0ff
Build id: 3.10.0-514.el7.x86_64 		0xef
8
run_ksoftirqd
vmlinux 0xffffffff8108f2c8
Build id: 3.10.0-514.el7.x86_64 		0x38
9
smpboot_thread_fn
vmlinux 0xffffffff810b90af
Build id: 3.10.0-514.el7.x86_64 		0x12f
10
? lg_double_unlock
vmlinux 0xffffffff810b8f80
Build id: 3.10.0-514.el7.x86_64 		0x90
11
kthread
vmlinux 0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64 		0xcf
12
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140
13
ret_from_fork
vmlinux 0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64 		0x58
14
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [ksoftirqd/1:13]
Modules linked in: xt_addrtype br_netfilter overlay xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 nfit snd_rawmidi snd_ac97_codec libnvdimm ppdev vmw_balloon intel_powerclamp ac97_bus sg coretemp snd_seq snd_seq_device snd_pcm snd_timer snd iosf_mbi soundcore crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr parport_pc parport i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom ata_generic
pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix scsi_transport_spi mptscsih libata e1000 mptbase i2c_core fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 13 Comm: ksoftirqd/1 Tainted: G        W    L ------------   3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff880184a82f10 ti: ffff880184a98000 task.ti: ffff880184a98000
RIP: 0010:[<ffffffffa00cc8ae>]  [<ffffffffa00cc8ae>] e1000_alloc_rx_buffers+0x36e/0x570 [e1000]
RSP: 0018:ffff880184a9bbd8  EFLAGS: 00000286
RAX: ffffc90000fc2818 RBX: ffff880184a9bbb8 RCX: 00000000000005f2
RDX: 00000000000000ad RSI: ffff88023065e180 RDI: ffff880184cc68c0
RBP: ffff880184a9bc30 R08: 0000000000019b00 R09: ffffffff8155bf77
R10: ffff880235659b00 R11: ffffea0007afb200 R12: ffffffff81aeae78
R13: ffff8801ebec9a00 R14: 000000002165ee9c R15: ffffffff815b0120
FS:  0000000000000000(0000) GS:ffff880235640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6a83b44c20 CR3: 00000001cb7a3000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
000005f284a9bc08 ffffffff8156f8a0 00000000000005f1 ffff880184cc6b70
ffff880235eea000 ffff880235eea098 0000000000000000 ffff880184cc68c0
ffff8800b8841ae0 ffff8801ebec9a00 ffff8800b8841af0 ffff880184a9bcd8
Call Trace:
[<ffffffff8156f8a0>] ? netif_receive_skb_internal+0x40/0xc0
[<ffffffffa00cc4a5>] e1000_clean_rx_irq+0x455/0x4f0 [e1000]
[<ffffffffa00ccd31>] e1000_clean+0x281/0x8f0 [e1000]
[<ffffffff815d16a0>] ? tcp_init_xmit_timers+0x30/0x30
[<ffffffff815700b0>] net_rx_action+0x170/0x380
[<ffffffff8108f0ff>] __do_softirq+0xef/0x280
[<ffffffff8108f2c8>] run_ksoftirqd+0x38/0x50
[<ffffffff810b90af>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f80>] ? lg_double_unlock+0x90/0x90
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 00 01 0f 1f 00 41 3b 5d 18 74 20 85 db 41 89 5d 18 8d 53 ff 0f 84 e0 01 00 00 0f ae f8 41 0f b7 45 36 49 03 87 d0 03 00 00 89 10 <48> 83 c4 30 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 48 8b 45