You’re right that best practice dictates that you don’t span your VLANs across multiple sites, however sometimes as you say, it is necessary. There’s really no “solution” to this, either you span your VLANs or you don’t. You must however try to keep it to a minimum.
What VXLAN provides is an underlay network that is composed of routers and switches, where in most cases, you will never have a VLAN span multiple sites. Site-to-site communication in the underlay network will take place using L3 communication with routing. However, the VXLAN in the overlay network, which is being tunneled over that underlay network, is still spanning multiple sites. So from the point of view of the overlay network, you still do have a broadcast domain (VLAN) that is spanning multiple sites. This cannot always be avoided, but it should be minimized as much as possible.
Typically in a campus network, you should be able to avoid it completely, however, it is in distributed datacenters that you most often need to span VLANs across sites. VXLAN doesn’t “solve” the issue, but it does provide you with a framework where you can more easily apply such a configuration. You should still do it sparingly…
In it you’ll find details about the headers and where they are added.
Remember that VXLANs use an overlay and an underlay network. The VXLAN header actually sits between the header information of the overlay and underlay networks. The underlay network carries the “tunneled” traffic and uses the UDP header with port 4789 or 8472 to carry that traffic.
Below you can see the headers involved with the implementation of the VXLAN.
The headers that are circled in red are those that belong to the underlay network. Like any network, it needs a layer 3 header and a layer 4 header, and that is why the UDP header is used there.
Thanks, Rene for these videos, Do you always have to map a VLAN to a VNI ? how do vxlan make use of these 16 Millions VNIs when i have to map a VLAN to a vni. Please let me understand i am really confused about it. Thanks
Remember that the purpose of VXLANs is not only to surpass the limitation of 4K VLAN IDs, but also to add flexibility to deployments, allowing for the simpler spanning of VLANs across sites. When there is no need to expand beyond that 4K limit, typical customer deployments will use a 1:1 mapping of available VLANs for simplicity.
However, when scalability to reach beyond this limit is needed, then it is possible to do so by assigning the same VLAN IDs to multiple unique VNIs found on different VTEPs within the fabric. So for example, you could have
VLAN 456 mapped to VNI 123 using a subnet of 10.1.1.0/24
VLAN 456 mapped to VNI 321 using a subnet of 10.2.2.0/24
The same VLAN ID is used, but on different VNIs. You can theoretically have 16 million VNIs each having 4000+ VLANs mapped to them.
Typically, you would have one VNI on a single physical switch, so you do have a limitation of 4k per physical device, but that is typically not a limiting factor.
Thanks so much for replying to this post is helpful but I have more question basically, is VNI and VTEP need to match on both sides? do you need a vtep for every VNI? can you map VINI to VNI with no VLAN involve? Your help is really appreciated. Thanks
When H1 sends its data, it is oblivious to any VXLAN VNI configuration. It sends its data, and when it reaches the VTEP, it is there that any VLAN IDs are added, based on the VLAN ID of the access port to which it is connected. (The VLAN ID may be added at a switch somewhere between the host and the VTEP if it exists). That VLAN ID information is encapsulated and mapped to a particular VNI.
Remember that VXLANs are at their most basic level, a tunneling or encapsulation mechanism. Whatever VLAN ID is in the frame, this will be encapsulated and used within the VXLAN infrastructure.
Cisco has an excellent and comprehensive document on VXLAN and its use of an MP_BGP EVPN underlay network. The following is an excerpt from that document which gives a brief but very clear understanding of how these technologies work together:
The initial IETF VXLAN standards (RFC 7348) defined a multicast-based flood-and-learn VXLAN without a control plane. It relies on data-driven flood-and-learn behavior for remote VXLAN tunnel endpoint (VTEP) peer discovery and remote end-host learning. The overlay broadcast, unknown unicast, and multicast traffic is encapsulated into multicast VXLAN packets and transported to remote VTEP switches through the underlay multicast forwarding. Flooding in such a deployment can present a challenge for the scalability of the solution. The requirement to enable multicast capabilities in the underlay network also presents a challenge because some organizations do not want to enable multicast in their data centers or WAN networks.
To overcome the limitations of the flood-and-learn VXLAN as defined in RFC 7348, organizations can use Multiprotocol Border Gateway Protocol Ethernet Virtual Private Network (MP-BGP EVPN) as the control plane for VXLAN. MP-BGP EVPN has been defined by IETF as the standards-based control plane for VXLAN overlays. The MP-BGP EVPN control plane provides protocol-based VTEP peer discovery and end-host reachability information distribution that allows more scalable VXLAN overlay network designs suitable for private and public clouds. The MP-BGP EVPN control plane introduces a set of features that reduces or eliminates traffic flooding in the overlay network and enables optimal forwarding for both west-east and south-north traffic.
The following is the link from this documentation from which you can find out more information:
If you have any more specific questions about this technology, let us know!
Great Lesson!
when VETP1 sends the packets to VETP2, it encapsulates them in another L2 header, that it is not mentioned for clarity. A lot of overhead
Yes, you are correct, there will be an additional layer 2 header added when information is sent from VTEP1 to VTEP2. However, in between these two devices, you may have multiple networks, routers, and even the Internet. So what L2 headers will be added depends upon the network topology between the two VTEPs. For this reason, only IP addresses are shown here, which will remain the same for the duration of the communication between the two VTEPs, whereas the L2 headers may change multiple times throughout that particular leg of the journey.
Yes, there is a lot of overhead! But as you can see, the more complex the overhead is, the simpler the the whole operation seems from the point of view of the hosts themselves. There is always a tradeoff…
Hi Rene,
VxLAN tutorial was excellent.
This is a suggestion! Not sure this is the right place.
I think it may be very helpful to have a lesson on EVPN. VxLAN EVPN goes almost hand-in-hand.
EVPN as technology can bring in extra values.
Great to hear that you enjoyed the lesson. Thanks so much for your suggestion, and yes it does make sense to include EVPN as part of the VXLAN topics. It would be great if you can make this suggestion on the Member Ideas page linked below. There you can share what kind of new topics you’d like to see covered on the site. You may find that others have made similar suggestions, and you can add your voice to theirs.
MP-BGP with EVPN is not a topic that is covered on the NetworkLessons site. However you can make a suggestion for new lesson topics on the following Member Ideas page.
There you can see what others have suggested, and you can add your voice to theirs.
In the meantime, take a look at the following post to find out more information about these features:
Hello Team
I have just joined the Network Lessons Course and I must the VXLAN is pretty easy to understand and explain now
I have 2 questions regarding the packet walk through section.
How did the H1 host came to know about the MAC address of H2?
How did switch came to know that the Packet I’m receving from H1 should go through the vxlan tunnel?
Great to hear that you are enjoying the content here and it is helpful for you. First of all, note that the IP addresses on the hosts in the lesson are incorrect. I have informed Rene to make the correction. Both H1 and H2 are on the same subnet, and should have addresses like so:
H1: 192.168.1.101
H2: 192.168.1.102
Now, on to your questions:
In the same way as if the two hosts were on the same physical network segment. ARP is used to determine this. In this particular case, the ARP request sent out by H1 is encapsulated within the VXLAN overlay network and is forwarded through the VXLAN infrastructure to reach H2. H2 responds with an ARP reply which in the same way traverses the VXLAN infrastructure encapsulated as usual and reaches H1. H1 then populates its ARP table.
This is achieved using the mapping information that VTEP1 has obtained. What mechanism is used to create these mappings? There are various options, as stated in the lesson. In the lesson, the VXLAN with multicast underlay is described in further detail. Once the VTEP sees the mapping it knows how to deal with the particular packet it has received, to encapsulate it and send it through the tunnel.
So, when the H1 sends the ARP our device being layer 3 device (I’m assuming) it should not have send the broadcast arp request from VTEP1 to VTEP2.
So, did VTEP1 generated the new broadcast to find the MAC of H2?
