That Other Big Project !

I've alluded to our other big summer project a couple of times, but now I'm finally going to give you the low-down on it !

When I returned from vacation in early January, I fully anticipated that the core of our network would remain fairly static until the summer of 2009. This is when, according to the 10-year network plan we had been developing (a topic for another post), we would do a "fork-lift" upgrade of the core. Since that was only 16 months away, I set to work developing requirements, scheduling initial discussions with vendors, etc. As part of this process, I also started meeting with various departments and groups to get a little better handle on what they would need - in addition to the UITS projects I already knew the network would have to support such as VoIP and IPTV. What I learned over the course of January and early February led to a slight change of plans :)

One of the important things I learned is that there are several departments looking to take advantage of the high-capacity, high-bandwidth networked storage systems we've deployed over the last 2 years. Our MDSS tape storage system now has 24 10GbE connected "front-end" servers each capable of moving data in and out of the system at around 3-4Gbps. The Data Capacitor is a disk-based storage system that also has 24 10GbE connected servers each capable at moving data between the IP network and disk at around 7Gbps. I met or spoke with at least 4 departments during January that were all looking to move large data sets between their buildings and these central storage systems at very high bandwidths.

Our current architecture aggregates all the 1GbE connections from the buildings into layer-2 ethernet switches, applies 802.1Q VLAN tags and trunks all those VLANs over 10GbE links to the routers. This architecture provides a lot of flexibility and works fine for large numbers of 1GbE connected buildings using a few hundred Mbps of bandwidth each, but not so well for a dozen 10GbE buildings bursting up to 3-4Gbps each. In addition, those layer-2 ethernet switches are also out of empty ports and modules.

The solutions to this are:

(1) Move the layer-3 routing function onto the aggregation switches that terminate the fiber connections to the buildings. This removes the bottleneck between the layer-2 aggregation switches and the layer-3 routing switches. It also frees up quite a few 4-port 10GbE modules that can be reused to support 10GbE connections to buildings.

(2) Upgrade the 16-port 1GbE modules to 24 or 48 port 1GbE modules. This frees up slots in the aggregation switches Inow layer-3 switches) to install the 4-port 10GbE modules to support 10GbE connections to buildings.

The other big thing that I learned about in January was PCI-DSS !! For those of you who haven't heard about PCI-DSS, that stands for Payment Card Industry - Data Security Standards. Think HIPAA on steroids for merchants that accept credit/debit cards :) PCI-DSS has a laundry list of network and processes requirements that must be met in order to be compliant.

As I dug deeper into what it would take to support the PCI-DSS requirements, it became clear (to me at least) that MPLS Layer-3 VPNs was the way to go. We had already been discussing MPLS VPNs for a while and several other universities have already deployed MPLS VPNs to solve problems like this. The general problem is that there are many different groups (or groups of systems), that each have unique network requirements and that have users/machines spread across many different buildings on campus. In addition to PCI-DSS compliant systems, you have building control systems (e.g. HVAC, security cameras, door access systems, etc), IP phones, and School of Medicine and Auxiliary Services that supports users/systems across many buildings. In a nutshell, MPLS Layer-3 VPN allows you to "group" these systems into separate virtual routers, each of which can have different network services and policies (firewall, NAT, IPS, etc).