I simply wasted a number of days making an attempt to determine tips on how to make the dozen (or so) platforms for which we applied VRRPv3 in netlab work collectively. That is the primary in a sequence of weblog posts describing the ridiculous stuff we found throughout that journey
The concept was fairly easy:
- Create a lab with the examined machine and a widely known probe related to the identical subnet.
- Disable VRRP (or interface) on the probe and test IPv4 and IPv6 connectivity by the examined machine (verifying it takes over possession of VRRP MAC and IP addresses).
- Reenable VRRP on the probe and alter its VRRP precedence a number of occasions to test the state transitions by INIT/BACKUP(decrease precedence)/MASTER(change in precedence)/BACKUP(preempting after a change in precedence).
When utilizing an Arista EOS VM because the well-known probe, I found that it refuses to yield to a preempting try from quite a few different units (for instance, Cisco IOS or Cisco Nexus OS) for the IPv4 tackle household. The identical units preempted EOS for the IPv6 tackle household.
Having two (or extra) VRRP masters on the identical phase can’t be good. If nothing else, you would possibly get duplicate MAC tackle or flapping MAC tackle messages on adjoining L2 switches, so it was time to determine what was occurring. I used tcpdump to see the VRRP packets and observed the dangerous vrrp cksum prognosis for the VRRP packets despatched by Cisco IOS:
VRRPv3 packet despatched by EOS appears OK; the one from Cisco IOS has a checksum error
07:56:11.666890 00:00:5e:00:01:d9 (oui IANA) > 01:00:5e:00:00:12 (oui Unknown), ethertype IPv4 (0x0800), size 46: (tos 0xc0, ttl 255, id 1, offset 0, flags [none], proto VRRP (112), size 32)
172.16.0.1 > vrrp.mcast.internet: VRRPv3, Commercial, vrid 217, prio 30, intvl 100cs, size 12, addrs: 172.16.0.42
07:56:11.743555 00:00:5e:00:01:d9 (oui IANA) > 01:00:5e:00:00:12 (oui Unknown), ethertype IPv4 (0x0800), size 60: (tos 0xc0, ttl 255, id 0, offset 0, flags [none], proto VRRP (112), size 32)
172.16.0.2 > vrrp.mcast.internet: VRRPv3, Commercial, vrid 217, prio 20, intvl 100cs, size 12, (dangerous vrrp cksum d87), addrs: 172.16.0.42
Likewise, Cisco IOS complained that the packets generated by Arista EOS comprise invalid checksum:
r2#debug vrrp packet
vrrp packet debugging enabled
r2#
*Jan 22 08:57:27.239: VRRPv4 Ethernet0/1 [217] vrrpv3 chksum D87
*Jan 22 08:57:27.239: VRRPv4 Ethernet0/1 [217] Ship V3 Commercial, Kind: 1, Group Id: 217, Precedence: 20, Advert interval: 100 csec, Depend: 1
*Jan 22 08:57:27.806: VRRP Ethernet0/1 Processing Packet:Invalid checksum Calculated chksum is nonzero (8CA0), Packet chksum is (76E6)
At that second, I made a decision it was excessive time for an additional journey into the RFC land. VRRPv3 is outlined in RFC 5798, which is obsoleted by RFC 9568 from April 2024. The latter RFC incorporates an fascinating change from the RFC 5798:
The checksum calculation in Part 5.2.8 has been clarified to specify exactly what’s included and that it doesn’t embrace the pseudo-header for IPv4.
That part is much more fascinating:
- For IPv4 messages, the checksum consists of solely the VRRP message.
- For IPv6, the checksum consists of the pseudo-header.
To recap:
- The wording in RFC 5798 was obscure, referring to pseudo-header being included within the checksum calculation.
- There is no such thing as a pseudo-header in IPv4.
- VRRPv3 implementations assumed both (A) ignore that wording as there is no such thing as a pseudo-header in IPv4 or (B) create pseudo-header for IPv4 out of skinny air (OK, utilizing the foundations from IPv6 RFC).
- Arista EOS and FRR appear to be the implementations utilizing the let’s pretend the pseudo-header method, whereas most others adopted the there is no such thing as a pseudo-header in IPv4 mentality.
- Most VRRP implementations ignore packets with incorrect checksum, probably leading to two VRRP masters on the identical phase in multi-platform deployments.
- RFC 5798 was revealed in 2010, and the primary implementations appeared within the early 2010s, but some distributors or open-source initiatives appear to have skipped the interoperability assessments with different platforms.
- The early implementations used the identical there is no such thing as a pseudo-header in IPv4 method, leaving one to marvel how we arrived on the two interpretations of the RFC (and the way the minority interpretation made it into tcpdump).
Lastly:
- Dell OS10 takes a very artistic method: it switches to the checksum calculation utilized by the opposite VRRP machine.
Happily, you may make Arista EOS RFC 9568-compliant with the vrrp ipv4 checksum pseudo-header exclude configuration command, and FRR has no vrrp checksum-with-ipv4-pseudoheader command since late 2022.
Nonetheless, as is usually the case, downstream distros can take a very long time to select up the adjustments. Cumulus Linux launch 5.10 nonetheless makes use of an older FRR model, and the present (as of January 2025) VyOS Vagrant field (v20240817.00.20) has no nerd knob to configure the underlying FRR VRRP course of.
