Info

Permalink
Function
__nf_conntrack_find_get
First occurrence
2020-10-02
Last occurrence
2020-10-04
State
NEW
Components
kernel
Quality
-43

Statistics

Operating system Count
CentOS 7 15
Architecture Count
x86_64 15
Related packages Count
kernel 15
 0:3.10.0-862.14.4.el7 6
 0:3.10.0-1062.18.1.el7 2
 0:3.10.0-1160.31.1.el7 2
 0:3.10.0-1160.76.1.el7 2
 0:3.10.0-1127.19.1.el7 1
 0:3.10.0-862.el7 1
 0:3.10.0-1160.119.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #1417533
Frame # Function Binary Source or offset Line
1
tcp_packet
nf_conntrack 0xffffffffc0800d70
Build id: 3.10.0-1160.119.1.el7.x86_64
-
2
? nf_conntrack_in
nf_conntrack 0xffffffffc07fa69d
Build id: 3.10.0-1160.119.1.el7.x86_64
0x3cd
3
ipv4_conntrack_in
nf_conntrack_ipv4 0xffffffffc08213c4
Build id: 3.10.0-1160.119.1.el7.x86_64
0x24
4
nf_iterate
vmlinux 0xffffffffa8cc0ef8
Build id: 3.10.0-1160.119.1.el7.x86_64
0x98
5
nf_hook_slow
vmlinux 0xffffffffa8cc0fe8
Build id: 3.10.0-1160.119.1.el7.x86_64
0xa8
6
ip_rcv
vmlinux 0xffffffffa8ccca19
Build id: 3.10.0-1160.119.1.el7.x86_64
0x339
7
? inet_del_offload
vmlinux 0xffffffffa8ccbf10
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
8
__netif_receive_skb_core
vmlinux 0xffffffffa8c7f669
Build id: 3.10.0-1160.119.1.el7.x86_64
0x739
9
__netif_receive_skb
vmlinux 0xffffffffa8c7f968
Build id: 3.10.0-1160.119.1.el7.x86_64
0x18
10
process_backlog
vmlinux 0xffffffffa8c8097e
Build id: 3.10.0-1160.119.1.el7.x86_64
0xae
11
net_rx_action
vmlinux 0xffffffffa8c80037
Build id: 3.10.0-1160.119.1.el7.x86_64
0x277
12
__do_softirq
vmlinux 0xffffffffa86a9595
Build id: 3.10.0-1160.119.1.el7.x86_64
0xf5
13
run_ksoftirqd
vmlinux 0xffffffffa86a9768
Build id: 3.10.0-1160.119.1.el7.x86_64
0x38
14
smpboot_thread_fn
vmlinux 0xffffffffa86d43fc
Build id: 3.10.0-1160.119.1.el7.x86_64
0x14c
15
? lg_double_unlock
vmlinux 0xffffffffa86d42b0
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
16
kthread
vmlinux 0xffffffffa86cb621
Build id: 3.10.0-1160.119.1.el7.x86_64
0xd1
17
? insert_kthread_work
vmlinux 0xffffffffa86cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
18
ret_from_fork_nospec_begin
vmlinux 0xffffffffa8dc51f7
Build id: 3.10.0-1160.119.1.el7.x86_64
0x21
19
? insert_kthread_work
vmlinux 0xffffffffa86cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 26s! [ksoftirqd/3:24]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter binfmt_misc overlay(T) vmw_vsock_vmci_transport vsock sunrpc nfit libnvdimm coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic vmwgfx drm_kms_helper syscopyarea pata_acpi sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix scsi_transport_spi
crct10dif_pclmul crct10dif_common mptscsih crc32c_intel serio_raw libata mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 24 Comm: ksoftirqd/3 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9709bafc5280 ti: ffff9709bafd8000 task.ti: ffff9709bafd8000
RIP: 0010:[<ffffffffc0800d70>]  [<ffffffffc0800d70>] tcp_packet+0x0/0xc40 [nf_conntrack]
RSP: 0018:ffff9709bafdbb18  EFLAGS: 00000246
RAX: ffffffffc0800d70 RBX: ffff9708f81f8f80 RCX: 0000000000000000
RDX: 0000000000000014 RSI: ffff9708fa8f2800 RDI: ffff970845a99540
RBP: ffff9709bafdbbc0 R08: 0000000000000002 R09: 0000000000000000
R10: ffff9709bafdbb80 R11: ffffffffc080df20 R12: ffffffffa8cf5b9d
R13: ffff9709bafdbaa8 R14: ffff970845b03e00 R15: ffffffffa8c6289d
FS:  0000000000000000(0000) GS:ffff970a7fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f682083a900 CR3: 0000000236e7c000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc07fa69d>] ? nf_conntrack_in+0x3cd/0x670 [nf_conntrack]
[<ffffffffc08213c4>] ipv4_conntrack_in+0x24/0x30 [nf_conntrack_ipv4]
[<ffffffffa8cc0ef8>] nf_iterate+0x98/0xe0
[<ffffffffa8cc0fe8>] nf_hook_slow+0xa8/0x110
[<ffffffffa8ccca19>] ip_rcv+0x339/0x420
[<ffffffffa8ccbf10>] ? inet_del_offload+0x40/0x40
[<ffffffffa8c7f669>] __netif_receive_skb_core+0x739/0xa20
[<ffffffffa8c7f968>] __netif_receive_skb+0x18/0x70
[<ffffffffa8c8097e>] process_backlog+0xae/0x180
[<ffffffffa8c80037>] net_rx_action+0x277/0x3a0
[<ffffffffa86a9595>] __do_softirq+0xf5/0x290
[<ffffffffa86a9768>] run_ksoftirqd+0x38/0x60
[<ffffffffa86d43fc>] smpboot_thread_fn+0x14c/0x1b0
[<ffffffffa86d42b0>] ? lg_double_unlock+0x40/0x40
[<ffffffffa86cb621>] kthread+0xd1/0xe0
[<ffffffffa86cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffffa8dc51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa86cb550>] ? insert_kthread_work+0x40/0x40
Code: 98 44 89 55 9c 41 29 c9 44 2b 4d a0 89 c1 31 c0 41 c1 e9 1f e8 b2 9a 1c e8 44 8b 5d 98 44 8b 55 9c e9 4b fa ff ff 0f 1f 44 00 00 <0f> 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 89 d5 41 54 49 
Complete report #825475
Frame # Function Binary Source or offset Line
1
nf_conntrack_in
nf_conntrack 0xffffffffc06a31e5
Build id: 3.10.0-1160.76.1.el7.x86_64
0x5
2
? ipv4_conntrack_in
nf_conntrack_ipv4 0xffffffffc06c83a4
Build id: 3.10.0-1160.76.1.el7.x86_64
0x24
3
nf_iterate
vmlinux 0xffffffffa3699aa8
Build id: 3.10.0-1160.76.1.el7.x86_64
0x98
4
nf_hook_slow
vmlinux 0xffffffffa3699b98
Build id: 3.10.0-1160.76.1.el7.x86_64
0xa8
5
ip_rcv
vmlinux 0xffffffffa36a5139
Build id: 3.10.0-1160.76.1.el7.x86_64
0x339
6
? inet_del_offload
vmlinux 0xffffffffa36a4660
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
7
__netif_receive_skb_core
vmlinux 0xffffffffa3659819
Build id: 3.10.0-1160.76.1.el7.x86_64
0x729
8
? __getnstimeofday64
vmlinux 0xffffffffa3107fbf
Build id: 3.10.0-1160.76.1.el7.x86_64
0x3f
9
__netif_receive_skb
vmlinux 0xffffffffa3659b18
Build id: 3.10.0-1160.76.1.el7.x86_64
0x18
10
netif_receive_skb_internal
vmlinux 0xffffffffa3659ba0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
11
napi_gro_receive
vmlinux 0xffffffffa365a828
Build id: 3.10.0-1160.76.1.el7.x86_64
0xd8
12
e1000_clean_rx_irq
e1000 0xffffffffc031145b
Build id: 3.10.0-1160.76.1.el7.x86_64
0x2bb
13
e1000_clean
e1000 0xffffffffc0312100
Build id: 3.10.0-1160.76.1.el7.x86_64
0x280
14
? nf_ct_delete
nf_conntrack 0xffffffffc06a102d
Build id: 3.10.0-1160.76.1.el7.x86_64
0x18d
15
net_rx_action
vmlinux 0xffffffffa365a1bf
Build id: 3.10.0-1160.76.1.el7.x86_64
0x26f
16
__do_softirq
vmlinux 0xffffffffa30a4d85
Build id: 3.10.0-1160.76.1.el7.x86_64
0xf5
17
run_ksoftirqd
vmlinux 0xffffffffa30a4f48
Build id: 3.10.0-1160.76.1.el7.x86_64
0x38
18
smpboot_thread_fn
vmlinux 0xffffffffa30ce9e4
Build id: 3.10.0-1160.76.1.el7.x86_64
0x144
19
? lg_double_unlock
vmlinux 0xffffffffa30ce8a0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
20
kthread
vmlinux 0xffffffffa30c5f91
Build id: 3.10.0-1160.76.1.el7.x86_64
0xd1
21
? insert_kthread_work
vmlinux 0xffffffffa30c5ec0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffa3799df7
Build id: 3.10.0-1160.76.1.el7.x86_64
0x21
23
? insert_kthread_work
vmlinux 0xffffffffa30c5ec0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 76s! [ksoftirqd/8:49]
Modules linked in: binfmt_misc tcp_lp nls_utf8 isofs xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) sunrpc sg ppdev virtio_balloon i2c_piix4 parport_pc parport pcspkr joydev ip_tables xfs libcrc32c sr_mod cdrom ata_generic virtio_blk pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata e1000 virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 8 PID: 49 Comm: ksoftirqd/8 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
task: ffff9746ac598000 ti: ffff9746ac5a0000 task.ti: ffff9746ac5a0000
RIP: 0010:[<ffffffffc06a31e5>]  [<ffffffffc06a31e5>] nf_conntrack_in+0x5/0x670 [nf_conntrack]
RSP: 0018:ffff9746ac5a3a58  EFLAGS: 00000293
RAX: ffff974d5d5ce000 RBX: 0000000000000000 RCX: ffff9745e8d05400
RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffffffffa3d16180
RBP: ffff9746ac5a3a60 R08: ffff9746ac5a3ae8 R09: ffffffffc06c8380
R10: 0000000000000000 R11: 000000000000afc6 R12: ffffffffa329d8a4
R13: ffff9746ac5a3a00 R14: ffffffffa329d25c R15: ffff9746ac5a39b8
FS:  0000000000000000(0000) GS:ffff974d5f400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0e0faa0338 CR3: 00000000352aa000 CR4: 00000000000006e0
Call Trace:
[<ffffffffc06c83a4>] ? ipv4_conntrack_in+0x24/0x30 [nf_conntrack_ipv4]
[<ffffffffa3699aa8>] nf_iterate+0x98/0xe0
[<ffffffffa3699b98>] nf_hook_slow+0xa8/0x110
[<ffffffffa36a5139>] ip_rcv+0x339/0x420
[<ffffffffa36a4660>] ? inet_del_offload+0x40/0x40
[<ffffffffa3659819>] __netif_receive_skb_core+0x729/0xa10
[<ffffffffa3107fbf>] ? __getnstimeofday64+0x3f/0xd0
[<ffffffffa3659b18>] __netif_receive_skb+0x18/0x60
[<ffffffffa3659ba0>] netif_receive_skb_internal+0x40/0xc0
[<ffffffffa365a828>] napi_gro_receive+0xd8/0x100
[<ffffffffc031145b>] e1000_clean_rx_irq+0x2bb/0x510 [e1000]
[<ffffffffc0312100>] e1000_clean+0x280/0x8e0 [e1000]
[<ffffffffc06a102d>] ? nf_ct_delete+0x18d/0x270 [nf_conntrack]
[<ffffffffa365a1bf>] net_rx_action+0x26f/0x390
[<ffffffffa30a4d85>] __do_softirq+0xf5/0x280
[<ffffffffa30a4f48>] run_ksoftirqd+0x38/0x50
[<ffffffffa30ce9e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffa30ce8a0>] ? lg_double_unlock+0x40/0x40
[<ffffffffa30c5f91>] kthread+0xd1/0xe0
[<ffffffffa30c5ec0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa3799df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa30c5ec0>] ? insert_kthread_work+0x40/0x40
Code: da 48 c7 c6 28 0c 6b c0 48 c7 c7 80 50 6b c0 31 c0 e8 a0 04 d1 e2 e9 b4 fe ff ff 90 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 <55> 48 89 e5 41 57 41 89 f7 41 56 41 55 49 89 fd 41 54 53 48 89 
Complete report #755806
Frame # Function Binary Source or offset Line
1
__nf_conntrack_find_get
nf_conntrack 0xffffffffc05f6d30
Build id: 3.10.0-1160.31.1.el7.x86_64
-
2
? nf_conntrack_in
nf_conntrack 0xffffffffc05f84c6
Build id: 3.10.0-1160.31.1.el7.x86_64
0x2e6
3
ipv4_conntrack_in
nf_conntrack_ipv4 0xffffffffc063b3a4
Build id: 3.10.0-1160.31.1.el7.x86_64
0x24
4
nf_iterate
vmlinux 0xffffffffa7096ab8
Build id: 3.10.0-1160.31.1.el7.x86_64
0x98
5
nf_hook_slow
vmlinux 0xffffffffa7096ba8
Build id: 3.10.0-1160.31.1.el7.x86_64
0xa8
6
ip_rcv
vmlinux 0xffffffffa70a21b9
Build id: 3.10.0-1160.31.1.el7.x86_64
0x339
7
? inet_del_offload
vmlinux 0xffffffffa70a16e0
Build id: 3.10.0-1160.31.1.el7.x86_64
0x40
8
__netif_receive_skb_core
vmlinux 0xffffffffa7056829
Build id: 3.10.0-1160.31.1.el7.x86_64
0x729
9
? __getnstimeofday64
vmlinux 0xffffffffa6b07a0f
Build id: 3.10.0-1160.31.1.el7.x86_64
0x3f
10
__netif_receive_skb
vmlinux 0xffffffffa7056b28
Build id: 3.10.0-1160.31.1.el7.x86_64
0x18
11
netif_receive_skb_internal
vmlinux 0xffffffffa7056bb0
Build id: 3.10.0-1160.31.1.el7.x86_64
0x40
12
napi_gro_receive
vmlinux 0xffffffffa7057838
Build id: 3.10.0-1160.31.1.el7.x86_64
0xd8
13
vmxnet3_rq_rx_complete
vmxnet3 0xffffffffc018f6b4
Build id: 3.10.0-1160.31.1.el7.x86_64
0x924
14
vmxnet3_poll_rx_only
vmxnet3 0xffffffffc01900e6
Build id: 3.10.0-1160.31.1.el7.x86_64
0x36
15
net_rx_action
vmlinux 0xffffffffa70571cf
Build id: 3.10.0-1160.31.1.el7.x86_64
0x26f
16
__do_softirq
vmlinux 0xffffffffa6aa4bc5
Build id: 3.10.0-1160.31.1.el7.x86_64
0xf5
17
run_ksoftirqd
vmlinux 0xffffffffa6aa4d88
Build id: 3.10.0-1160.31.1.el7.x86_64
0x38
18
smpboot_thread_fn
vmlinux 0xffffffffa6ace884
Build id: 3.10.0-1160.31.1.el7.x86_64
0x144
19
? lg_double_unlock
vmlinux 0xffffffffa6ace740
Build id: 3.10.0-1160.31.1.el7.x86_64
0x40
20
kthread
vmlinux 0xffffffffa6ac5e31
Build id: 3.10.0-1160.31.1.el7.x86_64
0xd1
21
? insert_kthread_work
vmlinux 0xffffffffa6ac5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffa7195df7
Build id: 3.10.0-1160.31.1.el7.x86_64
0x21
23
? insert_kthread_work
vmlinux 0xffffffffa6ac5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ksoftirqd/0:6]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_nat_ftp nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom
vmwgfx drm_kms_helper syscopyarea sysfillrect ata_generic sysimgblt pata_acpi fb_sys_fops ttm ahci drm sd_mod ata_piix libahci crc_t10dif nfit crct10dif_generic libnvdimm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.31.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff95b1baee5280 ti: ffff95b1baf28000 task.ti: ffff95b1baf28000
RIP: 0010:[<ffffffffc05f6d30>]  [<ffffffffc05f6d30>] __nf_conntrack_find_get+0x0/0x370 [nf_conntrack]
RSP: 0000:ffff95b1baf2ba48  EFLAGS: 00010a93
RAX: 0000000094d61989 RBX: ffff95b273268000 RCX: 0000000094d61989
RDX: ffff95b1baf2ba98 RSI: ffffffffa72a1a20 RDI: ffffffffa7715e40
RBP: ffff95b1baf2baf0 R08: 00000000d7a546e6 R09: 000000004bb3f40c
R10: ffff95b1baf2bab0 R11: ffffffffc060ad40 R12: ffff95b273268004
R13: ffffffffc02f6719 R14: ffff95b1baf2baf0 R15: ffffffffa6aa4047
FS:  0000000000000000(0000) GS:ffff95b27fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb08c98f6a0 CR3: 00000001eb0e6000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc05f84c6>] ? nf_conntrack_in+0x2e6/0x670 [nf_conntrack]
[<ffffffffc063b3a4>] ipv4_conntrack_in+0x24/0x30 [nf_conntrack_ipv4]
[<ffffffffa7096ab8>] nf_iterate+0x98/0xe0
[<ffffffffa7096ba8>] nf_hook_slow+0xa8/0x110
[<ffffffffa70a21b9>] ip_rcv+0x339/0x420
[<ffffffffa70a16e0>] ? inet_del_offload+0x40/0x40
[<ffffffffa7056829>] __netif_receive_skb_core+0x729/0xa10
[<ffffffffa6b07a0f>] ? __getnstimeofday64+0x3f/0xd0
[<ffffffffa7056b28>] __netif_receive_skb+0x18/0x60
[<ffffffffa7056bb0>] netif_receive_skb_internal+0x40/0xc0
[<ffffffffa7057838>] napi_gro_receive+0xd8/0x100
[<ffffffffc018f6b4>] vmxnet3_rq_rx_complete+0x924/0xef0 [vmxnet3]
[<ffffffffc01900e6>] vmxnet3_poll_rx_only+0x36/0xa0 [vmxnet3]
[<ffffffffa70571cf>] net_rx_action+0x26f/0x390
[<ffffffffa6aa4bc5>] __do_softirq+0xf5/0x280
[<ffffffffa6aa4d88>] run_ksoftirqd+0x38/0x50
[<ffffffffa6ace884>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffa6ace740>] ? lg_double_unlock+0x40/0x40
[<ffffffffa6ac5e31>] kthread+0xd1/0xe0
[<ffffffffa6ac5d60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa7195df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa6ac5d60>] ? insert_kthread_work+0x40/0x40
Code: ff ff 66 0f 1f 84 00 00 00 00 00 44 0f b7 ca 48 c7 c2 20 1a 2a a7 4c 01 ce 48 0f 45 d6 e9 24 ff ff ff f3 90 e9 e2 fc ff ff 66 90 <0f> 1f 44 00 00 55 48 89 e5 41 57 49 89 f7 41 56 49 89 fe 41 55 
Complete report #611356
Frame # Function Binary Source or offset Line
1
get_conntrack_index
nf_conntrack 0xffffffffc07d152d
Build id: 3.10.0-862.el7.x86_64
0x3d
2
tcp_packet
nf_conntrack 0xffffffffc07d317f
Build id: 3.10.0-862.el7.x86_64
0xcf
3
? local_bh_enable
vmlinux 0xffffffff81a99df7
Build id: 3.10.0-862.el7.x86_64
0x17
4
? __nf_conntrack_find_get
nf_conntrack 0xffffffffc07cb4ff
Build id: 3.10.0-862.el7.x86_64
0x17f
5
nf_conntrack_in
nf_conntrack 0xffffffffc07cd15d
Build id: 3.10.0-862.el7.x86_64
0x3bd
6
ipv4_conntrack_in
nf_conntrack_ipv4 0xffffffffc07f13a4
Build id: 3.10.0-862.el7.x86_64
0x24
7
nf_iterate
vmlinux 0xffffffff820273c8
Build id: 3.10.0-862.el7.x86_64
0x98
8
nf_hook_slow
vmlinux 0xffffffff820274b8
Build id: 3.10.0-862.el7.x86_64
0xa8
9
ip_rcv
vmlinux 0xffffffff82031d2b
Build id: 3.10.0-862.el7.x86_64
0x32b
10
? inet_del_offload
vmlinux 0xffffffff82031270
Build id: 3.10.0-862.el7.x86_64
0x40
11
__netif_receive_skb_core
vmlinux 0xffffffff81feb9f9
Build id: 3.10.0-862.el7.x86_64
0x729
12
? __getnstimeofday64
vmlinux 0xffffffff81af8c5f
Build id: 3.10.0-862.el7.x86_64
0x3f
13
__netif_receive_skb
vmlinux 0xffffffff81febd08
Build id: 3.10.0-862.el7.x86_64
0x18
14
netif_receive_skb_internal
vmlinux 0xffffffff81febd90
Build id: 3.10.0-862.el7.x86_64
0x40
15
napi_gro_receive
vmlinux 0xffffffff81fec9b8
Build id: 3.10.0-862.el7.x86_64
0xd8
16
e1000_clean_rx_irq
e1000 0xffffffffc034644b
Build id: 3.10.0-862.el7.x86_64
0x2bb
17
e1000_clean
e1000 0xffffffffc03470eb
Build id: 3.10.0-862.el7.x86_64
0x27b
18
? update_curr
vmlinux 0xffffffff81ad816c
Build id: 3.10.0-862.el7.x86_64
0x14c
19
? account_entity_dequeue
vmlinux 0xffffffff81ad4ade
Build id: 3.10.0-862.el7.x86_64
0xae
20
net_rx_action
vmlinux 0xffffffff81fec3af
Build id: 3.10.0-862.el7.x86_64
0x26f
21
__do_softirq
vmlinux 0xffffffff81a9a945
Build id: 3.10.0-862.el7.x86_64
0xf5
22
run_ksoftirqd
vmlinux 0xffffffff81a9ab08
Build id: 3.10.0-862.el7.x86_64
0x38
23
smpboot_thread_fn
vmlinux 0xffffffff81ac35e4
Build id: 3.10.0-862.el7.x86_64
0x144
24
? lg_double_unlock
vmlinux 0xffffffff81ac34a0
Build id: 3.10.0-862.el7.x86_64
0x40
25
kthread
vmlinux 0xffffffff81abae31
Build id: 3.10.0-862.el7.x86_64
0xd1
26
? insert_kthread_work
vmlinux 0xffffffff81abad60
Build id: 3.10.0-862.el7.x86_64
0x40
27
ret_from_fork_nospec_begin
vmlinux 0xffffffff8211f637
Build id: 3.10.0-862.el7.x86_64
0x21
28
? insert_kthread_work
vmlinux 0xffffffff81abad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ksoftirqd/0:3]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr nfit libnvdimm vmw_vmci shpchp i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix mptspi libata serio_raw scsi_transport_spi
e1000 mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3 Comm: ksoftirqd/0 Kdump: loaded Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9fd4fa5e9fa0 ti: ffff9fd4fa60c000 task.ti: ffff9fd4fa60c000
RIP: 0010:[<ffffffffc07d152d>]  [<ffffffffc07d152d>] get_conntrack_index+0x3d/0x50 [nf_conntrack]
RSP: 0018:ffff9fd4fa60f8f8  EFLAGS: 00000206
RAX: 0000000000000003 RBX: ffff9fd4f8b2d250 RCX: 0000000000000000
RDX: 0000000000000010 RSI: ffff9fd4f708e900 RDI: ffff9fd4824304e2
RBP: ffff9fd4fa60f8f8 R08: 0000000000000002 R09: 0000000000000014
R10: ffff9fd4fa60fa18 R11: ffffffffc07deec0 R12: ffff9fd4f8b2c000
R13: 0000000000000000 R14: 0000000000000000 R15: ffff9fd4b5a80c00
FS:  0000000000000000(0000) GS:ffff9fd4fb600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000006e6bc0 CR3: 000000004353c000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc07d317f>] tcp_packet+0xcf/0xa60 [nf_conntrack]
[<ffffffff81a99df7>] ? local_bh_enable+0x17/0x20
[<ffffffffc07cb4ff>] ? __nf_conntrack_find_get+0x17f/0x370 [nf_conntrack]
[<ffffffffc07cd15d>] nf_conntrack_in+0x3bd/0x660 [nf_conntrack]
[<ffffffffc07f13a4>] ipv4_conntrack_in+0x24/0x30 [nf_conntrack_ipv4]
[<ffffffff820273c8>] nf_iterate+0x98/0xe0
[<ffffffff820274b8>] nf_hook_slow+0xa8/0x110
[<ffffffff82031d2b>] ip_rcv+0x32b/0x410
[<ffffffff82031270>] ? inet_del_offload+0x40/0x40
[<ffffffff81feb9f9>] __netif_receive_skb_core+0x729/0xa20
[<ffffffff81af8c5f>] ? __getnstimeofday64+0x3f/0xd0
[<ffffffff81febd08>] __netif_receive_skb+0x18/0x60
[<ffffffff81febd90>] netif_receive_skb_internal+0x40/0xc0
[<ffffffff81fec9b8>] napi_gro_receive+0xd8/0x100
[<ffffffffc034644b>] e1000_clean_rx_irq+0x2bb/0x510 [e1000]
[<ffffffffc03470eb>] e1000_clean+0x27b/0x8e0 [e1000]
[<ffffffff81ad816c>] ? update_curr+0x14c/0x1e0
[<ffffffff81ad4ade>] ? account_entity_dequeue+0xae/0xd0
[<ffffffff81fec3af>] net_rx_action+0x26f/0x390
[<ffffffff81a9a945>] __do_softirq+0xf5/0x280
[<ffffffff81a9ab08>] run_ksoftirqd+0x38/0x50
[<ffffffff81ac35e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff81ac34a0>] ? lg_double_unlock+0x40/0x40
[<ffffffff81abae31>] kthread+0xd1/0xe0
[<ffffffff81abad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff8211f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff81abad60>] ? insert_kthread_work+0x40/0x40
Code: f6 c2 04 74 09 5d c3 0f 1f 80 00 00 00 00 f6 c2 02 75 1b f6 c2 01 b8 02 00 00 00 75 e8 83 e2 10 80 fa 01 19 c0 83 e0 02 83 c0 03 <5d> c3 90 c0 ea 04 89 d0 83 e0 01 5d c3 66 0f 1f 44 00 00 0f 1f 
Complete report #178854
Frame # Function Binary Source or offset Line
1
? local_bh_enable
vmlinux 0xffffffff9fe9d057
Build id: 3.10.0-862.14.4.el7.x86_64
0x17
2
? __nf_conntrack_find_get
nf_conntrack 0xffffffffc069a51f
Build id: 3.10.0-862.14.4.el7.x86_64
0x17f
3
nf_conntrack_in
nf_conntrack 0xffffffffc069c18d
Build id: 3.10.0-862.14.4.el7.x86_64
0x3cd
4
ipv4_conntrack_in
nf_conntrack_ipv4 0xffffffffc06d13a4
Build id: 3.10.0-862.14.4.el7.x86_64
0x24
5
nf_iterate
vmlinux 0xffffffffa042c568
Build id: 3.10.0-862.14.4.el7.x86_64
0x98
6
nf_hook_slow
vmlinux 0xffffffffa042c658
Build id: 3.10.0-862.14.4.el7.x86_64
0xa8
7
ip_rcv
vmlinux 0xffffffffa0436eeb
Build id: 3.10.0-862.14.4.el7.x86_64
0x32b
8
? inet_del_offload
vmlinux 0xffffffffa0436430
Build id: 3.10.0-862.14.4.el7.x86_64
0x40
9
__netif_receive_skb_core
vmlinux 0xffffffffa03f0b79
Build id: 3.10.0-862.14.4.el7.x86_64
0x729
10
__netif_receive_skb
vmlinux 0xffffffffa03f0e88
Build id: 3.10.0-862.14.4.el7.x86_64
0x18
11
netif_receive_skb_internal
vmlinux 0xffffffffa03f0f10
Build id: 3.10.0-862.14.4.el7.x86_64
0x40
12
napi_gro_receive
vmlinux 0xffffffffa03f1b38
Build id: 3.10.0-862.14.4.el7.x86_64
0xd8
13
vmxnet3_rq_rx_complete
vmxnet3 0xffffffffc0263729
Build id: 3.10.0-862.14.4.el7.x86_64
0x8a9
14
vmxnet3_poll_rx_only
vmxnet3 0xffffffffc0263d36
Build id: 3.10.0-862.14.4.el7.x86_64
0x36
15
net_rx_action
vmlinux 0xffffffffa03f152f
Build id: 3.10.0-862.14.4.el7.x86_64
0x26f
16
__do_softirq
vmlinux 0xffffffff9fe9dba5
Build id: 3.10.0-862.14.4.el7.x86_64
0xf5
17
run_ksoftirqd
vmlinux 0xffffffff9fe9dd68
Build id: 3.10.0-862.14.4.el7.x86_64
0x38
18
smpboot_thread_fn
vmlinux 0xffffffff9fec6774
Build id: 3.10.0-862.14.4.el7.x86_64
0x144
19
? lg_double_unlock
vmlinux 0xffffffff9fec6630
Build id: 3.10.0-862.14.4.el7.x86_64
0x40
20
kthread
vmlinux 0xffffffff9febdf21
Build id: 3.10.0-862.14.4.el7.x86_64
0xd1
21
? insert_kthread_work
vmlinux 0xffffffff9febde50
Build id: 3.10.0-862.14.4.el7.x86_64
0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffa05255f7
Build id: 3.10.0-862.14.4.el7.x86_64
0x21
23
? insert_kthread_work
vmlinux 0xffffffff9febde50
Build id: 3.10.0-862.14.4.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [swapper/4:0]
[<ffffffff9fe9d057>] ? local_bh_enable+0x17/0x20
[<ffffffffc069a51f>] ? __nf_conntrack_find_get+0x17f/0x370 [nf_conntrack]
[<ffffffffc069c18d>] nf_conntrack_in+0x3cd/0x670 [nf_conntrack]
[<ffffffffc06d13a4>] ipv4_conntrack_in+0x24/0x30 [nf_conntrack_ipv4]
[<ffffffffa042c568>] nf_iterate+0x98/0xe0
[<ffffffffa042c658>] nf_hook_slow+0xa8/0x110
[<ffffffffa0436eeb>] ip_rcv+0x32b/0x410
[<ffffffffa0436430>] ? inet_del_offload+0x40/0x40
[<ffffffffa03f0b79>] __netif_receive_skb_core+0x729/0xa20
[<ffffffffa03f0e88>] __netif_receive_skb+0x18/0x60
[<ffffffffa03f0f10>] netif_receive_skb_internal+0x40/0xc0
[<ffffffffa03f1b38>] napi_gro_receive+0xd8/0x100
[<ffffffffc0263729>] vmxnet3_rq_rx_complete+0x8a9/0xd50 [vmxnet3]
[<ffffffffc0263d36>] vmxnet3_poll_rx_only+0x36/0xb0 [vmxnet3]
[<ffffffffa03f152f>] net_rx_action+0x26f/0x390
[<ffffffff9fe9dba5>] __do_softirq+0xf5/0x280
[<ffffffff9fe9dd68>] run_ksoftirqd+0x38/0x50
[<ffffffff9fec6774>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff9fec6630>] ? lg_double_unlock+0x40/0x40
[<ffffffff9febdf21>] kthread+0xd1/0xe0
[<ffffffff9febde50>] ? insert_kthread_work+0x40/0x40
[<ffffffffa05255f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9febde50>] ? insert_kthread_work+0x40/0x40
Complete report #7081
Frame # Function Binary Source or offset Line
1
__nf_conntrack_find_get
nf_conntrack 0xffffffffc045bd94
Build id: 3.10.0-1127.19.1.el7.x86_64
0x64
2
nf_conntrack_in
nf_conntrack 0xffffffffc045d4c6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2e6
3
ipv4_conntrack_in
nf_conntrack_ipv4 0xffffffffc04dd3a4
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24
4
nf_iterate
vmlinux 0xffffffff84693cf8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x98
5
nf_hook_slow
vmlinux 0xffffffff84693de8
Build id: 3.10.0-1127.19.1.el7.x86_64
0xa8
6
ip_rcv
vmlinux 0xffffffff8469e7a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x339
7
? inet_del_offload
vmlinux 0xffffffff8469dcd0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
8
__netif_receive_skb_core
vmlinux 0xffffffff84653c19
Build id: 3.10.0-1127.19.1.el7.x86_64
0x729
9
__netif_receive_skb
vmlinux 0xffffffff84653f18
Build id: 3.10.0-1127.19.1.el7.x86_64
0x18
10
netif_receive_skb_internal
vmlinux 0xffffffff84653fa0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
11
netif_receive_skb
vmlinux 0xffffffff8465403c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1c
12
vmxnet3_rq_rx_complete
vmxnet3 0xffffffffc00f9639
Build id: 3.10.0-1127.19.1.el7.x86_64
0x8a9
13
vmxnet3_poll_rx_only
vmxnet3 0xffffffffc00fa0e6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x36
14
net_rx_action
vmlinux 0xffffffff846545bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x26f
15
__do_softirq
vmlinux 0xffffffff840a5695
Build id: 3.10.0-1127.19.1.el7.x86_64
0xf5
16
run_ksoftirqd
vmlinux 0xffffffff840a5858
Build id: 3.10.0-1127.19.1.el7.x86_64
0x38
17
smpboot_thread_fn
vmlinux 0xffffffff840cf0e4
Build id: 3.10.0-1127.19.1.el7.x86_64
0x144
18
? lg_double_unlock
vmlinux 0xffffffff840cefa0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
19
kthread
vmlinux 0xffffffff840c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
20
? insert_kthread_work
vmlinux 0xffffffff840c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
21
ret_from_fork_nospec_begin
vmlinux 0xffffffff84792d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
22
? insert_kthread_work
vmlinux 0xffffffff840c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [ksoftirqd/2:19]
Modules linked in: tcp_diag udp_diag inet_diag unix_diag ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 ip6table_filter nf_log_ipv4 nf_log_common xt_owner nf_nat_ftp xt_REDIRECT nf_nat_redirect xt_conntrack nf_conntrack_ftp xt_LOG xt_limit xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat nf_conntrack ip6_tables iptable_mangle iptable_raw ipt_REJECT nf_reject_ipv4 iptable_filter coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm mptspi libata scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common drm_panel_orientation_quirks mptbase crc32c_intel serio_raw vmxnet3 floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 19 Comm: ksoftirqd/2 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff98c6ba3f8000 ti: ffff98c6ba3f4000 task.ti: ffff98c6ba3f4000
RIP: 0010:[<ffffffffc045bd94>]  [<ffffffffc045bd94>] __nf_conntrack_find_get+0x64/0x370 [nf_conntrack]
RSP: 0018:ffff98c6ba3f7a10  EFLAGS: 00000246
RAX: ffff98c9697d53c8 RBX: ffff98c6ba3f79a0 RCX: ffff98c969780000
RDX: ffff98c6ba3f7a98 RSI: ffffffff848a1760 RDI: 000000000000aa79
RBP: ffff98c6ba3f7a40 R08: 000000004ee903c3 R09: 000000000857720d
R10: ffff98c6ba3f7ab0 R11: ffffffffc046ff20 R12: 0000000000000246
R13: 0000000000000001 R14: ffff98c97fde24a8 R15: ffff98c6b9c8f720
FS:  0000000000000000(0000) GS:ffff98c97fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000f74028 CR3: 00000001b999a000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc045d4c6>] nf_conntrack_in+0x2e6/0x670 [nf_conntrack]
[<ffffffffc04dd3a4>] ipv4_conntrack_in+0x24/0x30 [nf_conntrack_ipv4]
[<ffffffff84693cf8>] nf_iterate+0x98/0xe0
[<ffffffff84693de8>] nf_hook_slow+0xa8/0x110
[<ffffffff8469e7a9>] ip_rcv+0x339/0x420
[<ffffffff8469dcd0>] ? inet_del_offload+0x40/0x40
[<ffffffff84653c19>] __netif_receive_skb_core+0x729/0xa10
[<ffffffff84653f18>] __netif_receive_skb+0x18/0x60
[<ffffffff84653fa0>] netif_receive_skb_internal+0x40/0xc0
[<ffffffff8465403c>] netif_receive_skb+0x1c/0x70
[<ffffffffc00f9639>] vmxnet3_rq_rx_complete+0x8a9/0xef0 [vmxnet3]
[<ffffffffc00fa0e6>] vmxnet3_poll_rx_only+0x36/0xa0 [vmxnet3]
[<ffffffff846545bf>] net_rx_action+0x26f/0x390
[<ffffffff840a5695>] __do_softirq+0xf5/0x280
[<ffffffff840a5858>] run_ksoftirqd+0x38/0x50
[<ffffffff840cf0e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff840cefa0>] ? lg_double_unlock+0x40/0x40
[<ffffffff840c6691>] kthread+0xd1/0xe0
[<ffffffff840c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff84792d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff840c65c0>] ? insert_kthread_work+0x40/0x40
Code: 85 05 03 00 00 41 8b be e8 09 00 00 49 8b 8e f8 09 00 00 48 0f af 7d d0 48 c1 ef 20 3b 05 e1 3b 01 00 75 d3 48 8d 04 f9 48 8b 18 <f6> c3 01 74 1d e9 82 01 00 00 66 90 49 8b 86 28 0a 00 00 65 ff 
Complete report #4932
Frame # Function Binary Source or offset Line
1
__nf_conntrack_find_get
nf_conntrack 0xffffffffc05add91
Build id: 3.10.0-1062.18.1.el7.x86_64
0x61
2
nf_conntrack_in
nf_conntrack 0xffffffffc05af4c6
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2e6
3
ipv4_conntrack_in
nf_conntrack_ipv4 0xffffffffc053d3a4
Build id: 3.10.0-1062.18.1.el7.x86_64
0x24
4
nf_iterate
vmlinux 0xffffffffa4090438
Build id: 3.10.0-1062.18.1.el7.x86_64
0x98
5
nf_hook_slow
vmlinux 0xffffffffa4090528
Build id: 3.10.0-1062.18.1.el7.x86_64
0xa8
6
ip_rcv
vmlinux 0xffffffffa409af39
Build id: 3.10.0-1062.18.1.el7.x86_64
0x339
7
? inet_del_offload
vmlinux 0xffffffffa409a460
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
8
__netif_receive_skb_core
vmlinux 0xffffffffa404fbd9
Build id: 3.10.0-1062.18.1.el7.x86_64
0x729
9
? tcp4_gro_receive
vmlinux 0xffffffffa40c7600
Build id: 3.10.0-1062.18.1.el7.x86_64
0xf0
10
? __getnstimeofday64
vmlinux 0xffffffffa3b0790f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x3f
11
__netif_receive_skb
vmlinux 0xffffffffa404fed8
Build id: 3.10.0-1062.18.1.el7.x86_64
0x18
12
netif_receive_skb_internal
vmlinux 0xffffffffa404ff60
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
13
napi_gro_receive
vmlinux 0xffffffffa4050be8
Build id: 3.10.0-1062.18.1.el7.x86_64
0xd8
14
e1000_clean_rx_irq
e1000 0xffffffffc008a45b
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2bb
15
e1000_clean
e1000 0xffffffffc008b100
Build id: 3.10.0-1062.18.1.el7.x86_64
0x280
16
net_rx_action
vmlinux 0xffffffffa405057f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x26f
17
__do_softirq
vmlinux 0xffffffffa3aa5435
Build id: 3.10.0-1062.18.1.el7.x86_64
0xf5
18
run_ksoftirqd
vmlinux 0xffffffffa3aa55f8
Build id: 3.10.0-1062.18.1.el7.x86_64
0x38
19
smpboot_thread_fn
vmlinux 0xffffffffa3aced54
Build id: 3.10.0-1062.18.1.el7.x86_64
0x144
20
? lg_double_unlock
vmlinux 0xffffffffa3acec10
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
21
kthread
vmlinux 0xffffffffa3ac6321
Build id: 3.10.0-1062.18.1.el7.x86_64
0xd1
22
? insert_kthread_work
vmlinux 0xffffffffa3ac6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
23
ret_from_fork_nospec_begin
vmlinux 0xffffffffa418dd37
Build id: 3.10.0-1062.18.1.el7.x86_64
0x21
24
? insert_kthread_work
vmlinux 0xffffffffa3ac6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [ksoftirqd/0:6]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sb_edac coretemp ppdev iosf_mbi crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix libata drm mptspi crct10dif_pclmul scsi_transport_spi crct10dif_common
crc32c_intel mptscsih e1000 mptbase serio_raw drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Tainted: G             L ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
task: ffff97e2b8c05230 ti: ffff97e2b8c2c000 task.ti: ffff97e2b8c2c000
RIP: 0010:[<ffffffffc05add91>]  [<ffffffffc05add91>] __nf_conntrack_find_get+0x61/0x370 [nf_conntrack]
RSP: 0018:ffff97e2b8c2f970  EFLAGS: 00010246
RAX: ffff97e28b5c9fb8 RBX: ffff97e22c3c4400 RCX: ffff97e28b580000
RDX: ffff97e2b8c2f9f8 RSI: ffffffffa42a0fe0 RDI: 00000000000093f7
RBP: ffff97e2b8c2f9a0 R08: 00000000e3431889 R09: 00000000cbc962e5
R10: ffff97e2b8c2fa10 R11: ffffffffc05c1f20 R12: ffff97e22c3c4400
R13: ffffffffa4038702 R14: ffff97e2b8c2f8f8 R15: ffffffffa3bc6888
FS:  0000000000000000(0000) GS:ffff97e2bb200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff1d3995000 CR3: 00000000ba88a000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc05af4c6>] nf_conntrack_in+0x2e6/0x670 [nf_conntrack]
[<ffffffffc053d3a4>] ipv4_conntrack_in+0x24/0x30 [nf_conntrack_ipv4]
[<ffffffffa4090438>] nf_iterate+0x98/0xe0
[<ffffffffa4090528>] nf_hook_slow+0xa8/0x110
[<ffffffffa409af39>] ip_rcv+0x339/0x420
[<ffffffffa409a460>] ? inet_del_offload+0x40/0x40
[<ffffffffa404fbd9>] __netif_receive_skb_core+0x729/0xa10
[<ffffffffa40c7600>] ? tcp4_gro_receive+0xf0/0x1a0
[<ffffffffa3b0790f>] ? __getnstimeofday64+0x3f/0xd0
[<ffffffffa404fed8>] __netif_receive_skb+0x18/0x60
[<ffffffffa404ff60>] netif_receive_skb_internal+0x40/0xc0
[<ffffffffa4050be8>] napi_gro_receive+0xd8/0x100
[<ffffffffc008a45b>] e1000_clean_rx_irq+0x2bb/0x510 [e1000]
[<ffffffffc008b100>] e1000_clean+0x280/0x8e0 [e1000]
[<ffffffffa405057f>] net_rx_action+0x26f/0x390
[<ffffffffa3aa5435>] __do_softirq+0xf5/0x280
[<ffffffffa3aa55f8>] run_ksoftirqd+0x38/0x50
[<ffffffffa3aced54>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffa3acec10>] ? lg_double_unlock+0x40/0x40
[<ffffffffa3ac6321>] kthread+0xd1/0xe0
[<ffffffffa3ac6250>] ? insert_kthread_work+0x40/0x40
[<ffffffffa418dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa3ac6250>] ? insert_kthread_work+0x40/0x40
Code: a8 01 0f 85 05 03 00 00 41 8b be e8 09 00 00 49 8b 8e f8 09 00 00 48 0f af 7d d0 48 c1 ef 20 3b 05 e1 3b 01 00 75 d3 48 8d 04 f9 <48> 8b 18 f6 c3 01 74 1d e9 82 01 00 00 66 90 49 8b 86 28 0a 00