Two years ago, I’ve used multiple Intel X710-DA4 NICs to built an EVPN Setup for my company level66.network. Whilst most of the connections to other peers worked quite well, I’ve had a few links which showed some strange issues. Most commonly the links were reported as flapping from the other peers or they just reported packetloss on their end.

After quite some time of troubleshooting, switching optics and using different ports we found the solution. With the default configuration of the Linux driver, the NIC seems to sent out some (maybe malformed) LLDP messages causing issues with some specific networking gear. We were not able to narrow it down to a specific vendor, though. These messages are directly generated on the NIC and are therefore not shown in a tcpdump. That’s why we were not aware of them.

Once we’ve disabled LLDP in the driver, all issues were gone. This is the command to disable LLDP per port on the NIC, sadly there is no option to disable that feature as a whole.

sudo ethtool --set-priv-flags ens1f0 disable-fw-lldp on

The configuration has to be applied upon each boot. This is why we’ve added the statement to our ifupdown configuration besides increasing Rx/Tx-queue.

root@de-fra2-evpn2:~# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# Trunk for VXLAN termination
auto ens1f0
iface ens1f0 inet manual
    mtu 9000
    post-up /usr/sbin/ethtool --set-priv-flags $IFACE disable-fw-lldp on
    post-up /usr/sbin/ethtool -K $IFACE gro off gso off tso off
    post-up /usr/sbin/ethtool -G $IFACE tx 4096 rx 4096

Thank you @IPngNetworks for reminding me of that issue!

Edit (07.03.2023):

@WRMSR pointed out another option to me to disable the function on the NIC via a udev rule. The rule automatically applies the fix once the driver is initialized.

root@de-fra2-evpn2:~# cat /etc/udev/rules.d/disable_x710_fw_lldp.rules
ACTION=="add|change", SUBSYSTEM=="net", DRIVERS=="i40e", RUN="/usr/sbin/ethtool --set-priv-flags $name disable-fw-lldp on"

root@de-fra2-evpn2:~# udevadm control --reload-rules && udevadm trigger