VLAN overload? WIFI suddenly slowed down drastic after adding more devices

I am in a production environment with several vlan/wifi SSIDs. But my campus just went one-to-one with Chromebook devices and the production network that they are connected to, is the same vlan across the network. So we have 6 sites, all same Vlan. My question is will this cause a bottle neck? Before covid, we never had any issues. Today, first day back with all students having their own Chromebook so obviously double maybe triple typical device amount, that particular SSID took a drastic decrease in speed, sometimes not even auth to the network. It seemed to get stuck. Now we were able to narrow it down to that particular SSID/VLAN, by connecting the hung ones to hotspots and our guest network and normal network function was restored on those. Switch back to production and went back to be extremly laggy. It seems that DHCP has handed out over 7K IPs thus far and that is not all of them. Is that too much for one VLAN? My theory is that they are bottlenecking somewhere or taxing one resource due to too many devices but I am just not sure where. Any help or guidance on where to look will be extremely helpful.

We use a cisco WLC blade attached to our core in the MDF, and each site has a dist that is routing through this core at the MDF. We are also using cisco ASA

Single VLAN spanning across 6 sites?
7K devices from a single VLAN? If yes, you might want to segment your network to reduce the amount of Broadcast on your network that it might have.

It seems your production network is getting choked. Good starting point to troubleshoot this could be to connect a Laptop wired to your network and ping the default gateway, internal servers and the internet 1) With no users or very minimum users on the network. 2) With all the users on the network.

This can give you an idea regarding latency and packet loss (if any) that the network might be experiencing. Also, if you have a NMS server that might give you some information regarding bottlenecks or performance issues.

Regards,
Rahul

Hello Jesse

I would have to agree with @Rahul_K on this one. The best practice for network design dictates that you should limit your VLANs to specific physical locations. If you look at the following lesson, you will see the logic behind this limitation:

In your case, the problem is exacerbated because of the fact that you have spanned your VLANs across the WAN to 6 sites. This results in A LOT of unnecessary traffic traversing the WANs. This is further exacerbated by the fact that you have thousands of hosts on the same subnet. This means that broadcasts from a single host are reaching thousands upon thousands of end devices, only to be discarded.

Imagine a wireless host has just connected to the network. They send out a broadcast DHCP request packet. This packet is received by all 7000+ hosts on this single VLAN. That means ALL hosts must process these packets, discover they are not intended for them, and discard them. Network devices must relay these packets across all WANs and to all switches that include this VLAN. That’s a lot of wasted bandwidth as well as host processing power.

Many services use broadcasts including ARP, DHCP, as well as routing protocols such as EIGRP and OSPF. If you use multicast on your network, it too could be sent unnecessarily to all devices in the subnet.

The result is WAN links congested with unnecessary traffic, and hosts processing packets not intended for them. The network performance will be degraded.

The most immediate solution you can employ is to segment your network. Initially, at each location, create at least one VLAN for wired devices, and one for wireless, which means a total of 12 VLANs throughout your locations. Create a subnet for each VLAN such that the number of hosts within each subnet is reduced. The fewer the better. Anything below 250 hosts per subnet should be acceptable, but it depends upon the kind of usage of the network. You can further refine your design by examining the network performance.

I hope this has been helpful!

Laz

Thank you so much for you reply Lazaros! I should try to clarify as much as possible. We do have multiple vlans in our district but they are not per site. They do span across all 6 sites. However the DHCP scope is unique to each site.
The issue we are having right now is that our main Production SSID(the one that we add all staff, faculty, and student devices, only we have the password) tripled in the amount of devices that were on it, causing some sort of network bottle neck. Th3e other SSID is our guest networks using cisco ISE and different settings, was running fine, also the hardlined devices were running fine. So it was a only our main SSID where the amount of devices tripled. Those devices are using there own site subnet, but are tagged in the same vlan.

Hello Jesse

If a VLAN spans across all 6 sites, then any DHCP server within that VLAN will distribute IP addresses to devices on all 6 sites. There is no way to confine a DHCP scope to a particular site. As described, it sounds like there are six scopes operating on the same VLAN that spans six sites. This still means that broadcasts are reaching all the hosts as described in the previous post. The only way to resolve this is to create separate VLANs per site.

This seems to confirm it. Even if hosts are getting different IP addresses on different subnets at each site, the fact that you’re using a single VLAN still means that broadcasts are reaching all hosts on all sites. It seems that the network was sufficient to support these inefficiencies before the increase, but once the hosts and the resulting traffic increased, the network was no longer able to do so. The solution still remains in segmenting your network at the very least, with one VLAN per site for all of your wireless hosts, thus containing broadcast traffic within each physical site.

I hope this has been helpful!

Laz