lab06-VxLAN-L3
Task VxLAN. L3 VNI
Set up an Overlay using VXLAN EVPN to provide Layer 3 connectivity between host-clients.
We need to configure routing between host-client.
Network diagram
The lab is built using the following role-based device mapping host- linux, leaf - frr, spine - arista
Underlay IP Address Allocation
• Underlay addressing follows the format 10.x.y.z, where:
◦ x = Data Center ID
◦ y = Spine switch ID
◦ z = Sequential host address per leaf connection
• Host addressing uses the 172.16.x.z/24 format, where:
◦ x = Leaf switch ID
◦ z = Sequential host address
◦ Each leaf switch uses .1 as its own IP in the corresponding subnet
• Loopback addressing uses 192.168.a.b/32, where:
◦ a = 1 for spine switches
◦ a = 2 for leaf switches
◦ b = Spine or leaf ID (assigned sequentially)
• IPv6 addressing uses fd00::[IPv4]
Interconnect ipv4 ipv6
Spine-1
Eth1
10.1.1.0/31
fd00::10:1:1:0/127
Leaf-1
Eth1
10.1.1.1/31
fd00::10:1:1:1/127
Spine-1
Eth2
10.1.1.2/31
fd00::10:1:1:2/127
Leaf-2
Eth1
10.1.1.3/31
fd00::10:1:1:3/127
Spine-1
Eth3
10.1.1.4/31
fd00::10:1:1:4/127
Leaf-3
Eth1
10.1.1.5/31
fd00::10:1:1:5/127
Spine-2
Eth2
10.1.2.0/31
fd00::10:2:1:0/127
Leaf-1
Eth2
10.1.2.1/31
fd00::10:2:1:1/127
Spine-2
Eth2
10.1.2.2/31
fd00::10:2:1:2/127
Leaf-2
Eth2
10.1.2.3/31
fd00::10:2:1:3/127
Spine-2
Eth3
10.1.2.4/31
fd00::10:2:1:4/127
Leaf-3
Eth2
10.1.2.5/31
fd00::10:2:1:5/127
Host-1
Eth1
172.16.1.11/24
fd00::172:16:1:b/116
Leaf-1
Eth3
access vlan red
access vlan red
Host-2
Eth1
172.16.2.12/24
fd00::172:16:2:c/116
Leaf-2
Eth3
access vlan blue
access vlan blue
Host-3
Eth1
172.16.1.13/24
fd00::172:16:3:e/116
Leaf-3
Eth3
access vlan red
access vlan red
Host-4
Eth1
172.16.2.14/24
fd00::172:16:4:d/116
Leaf-3
Eth4
access vlan blue
access vlan blue
loopback
Spine-1
192.168.1.1
fd00::192:168:1:1
Spine-2
192.168.1.2
fd00::192:168:1:2
Leaf-1
192.168.2.1
fd00::192:168:2:1
Leaf-2
192.168.2.2
fd00::192:168:2:2
Leaf-3
192.168.2.3
fd00::192:168:2:3
Vlan-if
Leaf-1
1000
172.16.1.3/24
fd00::172:16:1:3/116
172.16.1.100/24
Leaf-2
1001
172.16.2.4/24
fd00::172:16:2:4/116
172.16.2.100/24
Leaf-3
1000
172.16.1.5/24
fd00::172:16:1:5/116
172.16.1.100/24
Leaf-3
1001
172.16.2.5/24
fd00::172:16:2:5/116
172.16.2.100/24
Network based on ospf(underlay)+ibgp(overlay). Area ospf 0, bgp as 65500
Launching a lab in the Netlab
There were no special launch requirements. To avoid mixing services and transport into a single namespace, I introduced a dedicated VRF called user, where both vlan-if interfaces were placed — allowing the hosts to communicate with each other. Since I typically work with a centralized gateway model (where L3 terminates on the border leaf), I decided to experiment with a distributed gateway to understand how it’s configured and how it forwards traffic. One caveat: the gateway module that handles FHRP protocols only stretches IPv4 addresses across devices. Attempts to extend this behavior to IPv6 weren’t very successful.
File Topology For quick reference, here’s the inline listing:
Validating
This is exactly the stage where the idea emerged to place the hosts (servers) into a dedicated VRF. This allows us to inspect and present routing tables without mixing service and transport domains. Based on the topology, all hosts should be able to reach each other, including the vlan interface on all switches.
There’s probably no need to run additional ping tests from other hosts — they all have mutual reachability and share a common routing table.
Here’s the routing table from leaf-3, which is the most illustrative since it hosts both VLANs.
Как видим у нас дублируются маршруты, происходит это потому что bgp evpn у нас поднят и на ipv4 и на ipv6. Уж не знаю поднимает ли кто фабрику с dual-stack evpn, но в целом идея интересная. Надо бы попробовать собрать смешанную фабрику где spine-1 ipv4 only, spine-2 ipv6 only, но отложим эксперимент на другой раз. Ниже выводы подтверждающие тезис о дублировании
As we can see, route entries are duplicated — this happens because BGP EVPN is running over both IPv4 and IPv6. I’m not sure if anyone actually deploys a dual-stack EVPN fabric in production, but conceptually it’s an interesting idea. It might be worth experimenting with a mixed fabric, where spine-1 runs IPv4-only and spine-2 runs IPv6-only — though we’ll save that for another time. Below are the outputs confirming the route duplication caused by dual-stack EVPN advertisements.
Everything is back in place. By the way, EVPN IPv6 peering didn’t work out-of-the-box on Arista devices — I had to manually adjust the Arista EVPN template: usr/local/lib/python3.10/dist-packages/netsim/ansible/templates/evpn/eos.j2
Now that routing between hosts is working — which was the main goal — let’s take a look at how the fabric sees the EVPN state.
Here’s what spine-1 reports about our EVPN control plane:
So we’ve got three VNIs - 101000, 101001 l2 domain, и 10000 bind vrf user. This setup confirms we’re using a symmetric IRB model.
Here’s what we observe in Wireshark on leaf-3:
When communication occurs within the same L2 domain, traffic is encapsulated using a single VXLAN VNI (e.g.,
101000or101001).When traffic crosses between different VLANs, a different VNI is used — specifically the L3 VNI (e.g.,
10000), which is bound to theip-vrf..We only instantiate
vlan-ifinterfaces and their associated L2 VNIs on the leafs directly connected to hosts.For inter-VLAN communication, we rely on the IP VRF and its associated L3 VNI, which handles routing between VXLAN segments.
evpn routing table from leaf-1
In that output, we can clearly see:
• Type 2 routes — MAC/IP advertisements for endpoints within the L2 VNI (host reachability).
• Type 3 routes — Inclusive multicast routes used for BUM (broadcast, unknown unicast, multicast) replication within the L2 domain.
• Type 5 routes — IP prefix routes associated with the L3 VNI, enabling inter-VLAN routing via the symmetric IRB model.
Device configuration files:
Last updated