In the last post of this series we introduced how Brocades Ethernet fabrics function from a high level and we discussed the need for it. I promised that in this post we would begin to discuss how the Brocade VDX switch works in it’s two operational modes, Classic Mode and VCS mode. We will not be spending much time on the Classic Mode, mainly because most already understand this mode. In Classic mode you operate as a standard ethernet switch. This means things like Layer 2 forwarding, MAC learning, STP and so on are in the mix. Since we want to get away from that mix lets focus on VCS mode.

VCS mode has many of the capabilities of a Classic Switch but there are some additions. These additions include Transpared LAN Service, vLAGs, Distributed Config Management, and End-to-End FCoE. We will discuss these features as we make our way through. For now, recall that in the Part 1 post we talked about how a cluster is formed and we had edge ports that connect to non-VCS mode switches and internally we have Fabric ports negotiated between switches that operate in VCS mode. All of the fabric ports were transparent to those on the outside of the cluster. The determining factor on whether a switch operates in VCS mode is a setting that is user controlled. If a switch comes up with the VCS mode set to OFF it assumes the Classic Switch mode and all the ports would be defaulted to a shutdown state.

If the a switch is configured with VCS mode enabled when it powers up then all the ports transition from shutdown state and start operating as edge ports. Next, the Brocade Proprietary protocol called FLDP, or Fabric Link Discovery Protocol, discovers other switches that are operating in VCS mode and if the neighbor is a Brocade VDX switch. An illustration of this is seen in the image below.

If the neighbor is not a VDX switch the port will become an edge port. If the neighbor is a VDX Switch we then want to know if its operating in VCS mode. If it is operating in VCS mode the VCS ID must match. Each VCS fabric is identified by a VCS ID which defaults to 0. Assuming that the neighboring VDX switch has the same VCS ID the port transitions to a fabric port and an ISL is established.

Brief Mention of TRILL

Trill is used within the Fabric (with some proprietary Brocade functions) and every switch assumes itself to be a principle RBridge. The RBridge ID’s must be unique and the principle Rbridge determines if this is happening. If there is a conflict then the principle switch keeps the joining switch with a duplicate RBridge ID segmented. At this point the RBridge ID on one of the conflicting switches needs to be changed and if the ID is changed the switch needs to be rebooted for it to take effect.

Control Plane VLANs

There are two control plane VLANS used for control traffic. There is an Edge Control VLAN (4095) and a Fabric Control VLAN (4093). These are both created upon boot up of the switch. The edge control VLAN is what tunnels control frames between two edge ports across the fabric. Recall in the last post we mentioned that BPDU’s were transparently passed across the fabric. The Edge Control VLAN is where that happens. Control traffic on the Fabric Control VLAN is never sent out of edge ports and is used for internal VCS control frames.

Now that we have a basic understanding of how VCS-enabled switches establish a Fabric we will close this post out. In the next VCS post we will discuss Layer 2 forwarding.

To begin understanding Ethernet Fabrics, as seen through the eyes of Brocade, it’s important to know about classic ethernet switches, how they have traditionally been provisioned, and why the way we have done this in the past isn’t really the best way to do things.
Let’s begin with figure 1 where we see a classic ethernet switch topology. In the topology let’s assume that we have traffic flowing between Switch A and Switch E. What we know about traditional technology is that for Layer 2 we cannot have a loop. Why? Because loops are bad. Why are they bad? Because of the design of ethernet. So, with that in mind, we are to understand that we have a loop between B, C, and D, and that somehow that loop will be blocked. How so? Spanning Tree Protocol. It doesn’t matter if its Rapid, Multiple, or any other flavor. The fact is that it’s going to be running and it’s going to remove the loop.

So now looking at figure 2 we can see the topology now that spanning tree has done its thing.

What’s the big deal?

Why doesn’t this work well in todays networks? Because the convergence time when STP flips links takes too long and you wind up losing packets, and because you’re wasting a perfectly good link.

So how does an Ethernet Fabric work and why does it solve my problems?

At a high level, an Ethernet fabric provides a more flexible interconnected network between individual switches. We’ll just call this a Fabric. Switches that form a fabric create something called a virtual cluster which consists of physical switches. This is seen in figure 3, where switches B, C, and D have formed a virtual cluster, or fabric. You can now see how the classic ethernet switches, A and E, are sitting outside of the virtual cluster, connected to the cluster by an edge port, and the Inter-Switch Links of the cluster are now called Fabric Ports.

Fabric Ports are transparent to the classic ethernet switches which are connected to edge ports. The fabric and fabric ports appear as just a single switch to external classic ethernet switches. This is reminisce of how MST domains are seen to CST if you’re paying attention. A key difference however, is that we are no longer running STP in the Virtual Cluster.

So what about the classic ethernet switches that are still running STP? Now, BPDU’s used by the classic ethernet switches, are transported transparently through the Fabric. Understanding these basic concepts bring us to the point where we need to begin looking at how a switch operates and how a cluster is formed. In the next post we’ll take a look at the Brocade VDX switch and it’s two operational modes, Classic Mode and VCS mode.