lab08-VxLAN&M-Routing
Task VxLAN. Routing.
To advertise aggregated prefixes via EVPN Route Type 5, you need to configure IP prefix summarization within the EVPN control plane.
Place two "clients" in separate VRFs within the same EVPN fabric. Configure inter-VRF routing between the clients via an external device (e.g., border router, firewall, or similar).
Network diagram
We’re keeping Arista EOS as the Leaf platform so we can retain a portion of the MLAG setup between leaf-1 and leaf-2. leaf-4 will act as the border leaf, and FRR will be used as the external router. For client devices, we’ll stick with Linux hosts using bonded interfaces.
The lab is built using the following role-based device mapping host- linux, leaf - arista, spine - arista
Underlay IP Address Allocation
• Underlay addressing follows the format 10.x.y.z, where:
◦ x = Data Center ID
◦ y = Spine switch ID
◦ z = Sequential host address per leaf connection
• Host addressing uses the 172.16.x.z/24 format, where:
◦ x = Leaf switch ID
◦ z = Sequential host address
◦ Each leaf switch uses .1 as its own IP in the corresponding subnet
• Loopback addressing uses 192.168.a.b/32, where:
◦ a = 1 for spine switches
◦ a = 2 for leaf switches
◦ b = Spine or leaf ID (assigned sequentially)
• IPv6 addressing uses fd00::[IPv4]
Interconnect ipv4 ipv6
Spine-1
Eth1
10.1.1.0/31
fd00::10:1:1:0/127
Leaf-1
Eth1
10.1.1.1/31
fd00::10:1:1:1/127
Spine-1
Eth2
10.1.1.2/31
fd00::10:1:1:2/127
Leaf-2
Eth1
10.1.1.3/31
fd00::10:1:1:3/127
Spine-1
Eth3
10.1.1.4/31
fd00::10:1:1:4/127
Leaf-3
Eth1
10.1.1.5/31
fd00::10:1:1:5/127
Spine-1
Eth4
10.1.1.6/31
fd00::10:1:1:6/127
Leaf-4
Eth1
10.1.1.7/31
fd00::10:1:1:7/127
Spine-2
Eth1
10.1.2.0/31
fd00::10:1:2:0/127
Leaf-1
Eth2
10.1.2.1/31
fd00::10:1:2:1/127
Spine-2
Eth2
10.1.2.2/31
fd00::10:1:2:2/127
Leaf-2
Eth2
10.1.2.3/31
fd00::10:1:2:3/127
Spine-2
Eth3
10.1.2.4/31
fd00::10:1:2:4/127
Leaf-3
Eth2
10.1.2.5/31
fd00::10:1:2:5/127
Spine-2
Eth4
10.1.2.6/31
fd00::10:1:2:6/127
Leaf-4
Eth2
10.1.2.7/31
fd00::10:1:2:7/127
Leaf-1
Eth3
10.0.1.0/31
fd00::10:0:1:0/127
Leaf-2
Eth3
10.0.1.1/31
fd00::10:0:1:1/127
Leaf-1
Eth4
------------
------------
Leaf-2
Eth4
------------
------------
Leaf-4
Eth5 vrf-red
10.0.5.0/31
fd00::10:0:5:0/127
R-1
Eth1
10.0.5.1/31
fd00::10:0:5:1/127
Leaf-4
Eth6 vrf-blue
10.0.5.2/31
fd00::10:0:5:2/127
R-1
Eth2
10.0.5.3/31
fd00::10:0:5:3/127
Host-1
Eth1
172.16.1.11/24
fd00::172:16:1:b/116
Leaf-1
Eth5
access vlan red
access vlan red
Host-1
Eth2
172.16.1.11/24
fd00::172:16:1:b/116
Leaf-2
Eth5
access vlan red
access vlan red
Host-2
Eth1
172.16.1.12/24
fd00::172:16:1:c/116
Leaf-1
Eth6
access vlan blue
access vlan blue
Host-2
Eth2
172.16.1.12/24
fd00::172:16:1:с/116
Leaf-2
Eth6
access vlan blue
access vlan blue
Host-3
Eth1
172.16.2.13/24
fd00::172:16:2:d/116
Leaf-3
Eth3
access vlan red
access vlan red
Host-4
Eth1
172.16.2.14/24
fd00::172:16:2:e/116
Leaf-3
Eth4
access vlan blue
access vlan blue
Host-5
Eth1
172.16.2.15/24
fd00::172:16:2:f/116
Leaf-4
Eth3
access vlan red
access vlan red
Host-6
Eth1
172.16.2.16/24
fd00::172:16:2:10/116
Leaf-4
Eth4
access vlan blue
access vlan blue
loopback
Spine-1
192.168.1.1
fd00::192:168:1:1
Spine-2
192.168.1.2
fd00::192:168:1:2
Leaf-1
192.168.2.1
fd00::192:168:2:1
Leaf-2
192.168.2.2
fd00::192:168:2:2
Leaf-1
192.168.2.0
fd00::192:168:2:0
Leaf-2
192.168.2.0
fd00::192:168:2:0
Leaf-3
192.168.2.3
fd00::192:168:2:3
Leaf-4
192.168.2.4
fd00::192:168:2:4
R-1
192.168.3.1
fd00::192:168:3:1
Network based on ospf(underlay)+ibgp(overlay). Area ospf 0, bgp as 65500, external bgp as 65100
Launching a lab in the Netlab (l2 mode)
Since the assignment requires careful reading, we ended up building two lab topologies during execution. The first attempt was based on Layer 2, and host-to-host routing worked as expected.
r1 was deployed using a router-on-a-stick design — a classic setup where a single physical interface handles multiple VLANs via subinterfaces. Diagram bellow
File Topology For quick reference, here’s the inline listing:
And it worked fine. The only concern was the absence of EVPN Type-5 routes — but of course, there was no reason for them to appear: None of the Leaf switches were involved in inter-VRF routing, and in fact, they had no awareness of the VRFs at all. Realizing the design was off, I went back and started building it the right way.
Launching a lab in the Netlab (l3 mode)
Correct topology for the assignment. We decided to implement a distributed gateway architecture, where each VRF domain has its own gateway on every leaf switch. Some nuances emerged during route exchange between leaf-4 and r-1, particularly across VRFs — but we’ll cover that later. The hard-earned configuration file is available below the diagram.
Validating
Connectivity is verified using the Netlab validate autotest. If everything is configured correctly, all endpoints will successfully respond to ping tests.
Now let’s talk about the nuances of route exchange through R-1. If you simply launch the lab, both VRFs will receive the loopback route from R-1, and R-1 will have visibility into all networks. However, the VRFs won’t see each other’s routes, because they share the same BGP AS number — and by default, BGP does not install routes received via eBGP that originate from its own AS.
We explored several options to resolve this:
• bgp allowas-in on the peer — allows the router to accept routes from its own AS.
Oddly, this didn’t work on Arista, though it did on FRR. Possibly a missed config detail.
• route-map that strips the BGP AS from routes received from leaf-4 — this worked perfectly.
• creating separate VRFs on R-1 and linking them via another IGP protocol — effectively removes BGP attributes, but adds complexity.
Functionally similar to the route-map approach, but more convoluted.
• default-originate — surprisingly effective and simple, though it didn’t come to mind immediately.
The result resembles hierarchical VPN (HVPN) in Huawei’s terminology.
• bgp override-as — again, didn’t work on Arista, but FRR accepted it.
• Using different AS numbers for each VRF — technically valid, but conventionally, VRFs tend to share the AS of the router they reside on. I briefly considered running leaf-4 on FRR while keeping the rest of the fabric on Arista — but VXLAN VNI mappings started behaving inconsistently. Spent a lot of time trying to stitch it all together, but ultimately decided against submitting the lab with such workarounds.
Now, let’s talk about how to make everything work cleanly and reliably.
Initially, I went with the default route advertisement approach — it worked well in Netlab. Just a single line: bgp.default_originate: True, and the default route is advertised in the session. However, I later discovered issues with MLAG behavior on leaf-1 and leaf-2 — specifically, duplicate packets appearing in ping results. The root cause: both leaf-1 and leaf-2, as part of the same fabric, must send BGP updates from the same source address. To achieve this, they need to have a loopback interface with the same IP address configured on both devices. Since the overarching goal of this lab series is to avoid using configure terminal manually, we need to inject this configuration during device initialization.
Ideally, I’d write a custom Netlab module to handle this — but due to time constraints, I opted for a shortcut: Using the config option in the topology file, which allows injecting additional configuration snippets after the lab is initialized. And since we’re already adding custom config for leaf-1 and leaf-2, we can also include the route-map configuration for R-1 at the same time.
Important note: For some reason, I had the idea that the same VNI could be used for both L2 and L3 services — but in practice, that’s not the case at all. This misconception led to a lot of wasted time trying to trace routes that simply wouldn’t propagate.
Now, let’s take a look at the routing tables.
Note that the routes received from the leaf-1 / leaf-2 pair are advertised with the source address 192.168.2.0/32, which is shared between both leaf switches.
vrf routing tables
The loopback from R-1 isn’t being received because it’s filtered out by the route-map.
Leaf-4 vrf routing table
By the way, I forgot to mention a fantastic Netlab add-on tool that helps avoid confusion about how things are wired — Graphite. It provides a clear visualization of interface-level connectivity, showing exactly how devices are linked. Even better, it allows you to SSH into multiple devices simultaneously, which I find extremely convenient — especially if you’re not a fan of tmux.
Some capture from leaf-4
EVPN Type 5 routes advertised to spines
EVPN Type 3 routes
Looking at the ICMP traffic on leaf-4, we observe that:
• The incoming packet arrived from the fabric via VNI 10000, which corresponds to VRF red using EVPN Type 5 (IP prefix route).
• The reply packet, however, was sent out via VNI 10000 mapped to VLAN 1000 using EVPN Type 3 (MAC/IP advertisement).
Clearly, something got mixed up — likely a VNI-to-VRF or VLAN mapping inconsistency, or a misinterpretation of how Type 5 and Type 3 routes are being handled.
The easiest way to observe traffic flow across the fabric turned out to be L3 ping tests between hosts connected to it in different VRFs.Here’s what we see:
1. A request from h3 is sent into the fabric
2. That same request returns from the fabric
3. A reply from h4 is sent into the fabric
4. The reply arrives back from the fabric
In the reverse direction, the same pattern is observed — but over IPv6.
This confirms that inter-VRF routing via the fabric is functioning, and that Type 5 route propagation is correctly enabling bidirectional communication.
If we look at Layer 2 connectivity, we’ll see that ping traffic within the fabric travels over isolated VXLAN segments.
1,2 VRF red; 3,4 VRF blue. That wraps up the lab setup.
Device configuration files:
Last updated