That pretty much sums up the last 4 weeks of my life. GENI Solicitation 3 proposals are due by 5pm tomorrow. The IU GlobalNOC is leading or partnering on several different proposals for various parts of the solicitation. I'm personally working on 2 proposals, PI on one and Co-PI on the other.
I'm looking forward to getting back to "normal" after tomorrow and there's no shortage of other work to be done. We're evaluating our options for 2011 router refreshes for both IU and I-Light. Our pilot of the Summer of Networking internship program went extremely well and we're already preparing to continue, improve and hopefully expand the program for next year. We're also working on a plan to expand the hands-on network training opportunities for networking staff both at IU and other universities. GEC9 (9th GENI Engineering Conference) will be here before you know it and I suspect the preparations will kick into high gear after proposals are submitted tomorrow !
Thursday, August 19, 2010
Tuesday, July 13, 2010
Openflow @ Internet2 Joint Techs
Internet2 is holding their summer Joint Techs Workshop at Ohio State this week and Openflow was featured prominently on yesterday's afternoon's agenda. At 3:00 Srini Seeththaraman from Stanford gave an excellent overview of Openflow. I followed that up with a talk at 4:30 that was focused on the practical aspects of potential Openflow applications in R&E network and what network engineers can do to get started. That was immediately followed by a presentation from Heidi Picher Dempsey from the GENI Project Office who talked about GENI and Openflow's application within the GENI infrastructure. GENI and Openflow were also the primary topic among the regional networks at the Gigapop Geeks BoF with both Heidi and I leading discussions. There were many good questions and excellent discussion about Openflow and GENI.
The slides and archived video from all of the presentations is available on the Internet2 Joint Techs Workshop agenda page:
http://tinyurl.com/24rzjn5
The slides and archived video from all of the presentations is available on the Internet2 Joint Techs Workshop agenda page:
http://tinyurl.com/24rzjn5
Thursday, June 17, 2010
Openflow Wish List
There are very smart people involved in the development of Openflow. However, I suspect very few of them actively manage networks on a day-to-day basis. Now that the code is in the hands of network engineers, we can see what's needed to actually get this running in production networks.
When it comes to emerging technologies, this space between the development and actual production use - between developers and the network engineers in the trench - is something I find incredibly interesting. It's great to be involved in the development at the point you can provide substantive feedback into the actual product or technology.
And that is where we are today with Openflow. We have Openflow deployed to 4 "production" switches and have a wireless SSID in 3 buildings across campus that feeds into an Openflow switch. The cool thing is that it all pretty much works. The problem is that, when it doesn't work, it's a pretty big pain to figure out why. Yesterday I compared it to the early days of the GSRs when the tables on one of the linecards would get out of sync, but it's a bit worse because the "linecards" are spread across the whole campus and there are very few debugging tools available.
There are a number of debugging features that would be useful, but I think the most useful one would be a way to see the dataplane and control-plane packets at the same time. One way to do this would be for the switch vendors to allow you to add Openflow control-plane packets into a port-mirroring configuration. This would allow me to hook a sniffer up to a switch port and mirror both the traffic to/from a switch port and the Openflow control messages to the sniffer.
Why would this be useful ? One problem we're having right now is that some laptops take 1-2 minutes to get a DHCP lease on the Openflow network. Is the switch taking a long time to encapsulate the first DHCP message into an Openflow message and send it to the controller ? Is the controller taking a long time to send the packet-out and flow-add messages to the switch ? Are the Openflow messages getting lost along the way ? Today I have to run a tcpdump on the Openflow controller to capture control-plane packets and Wireshark on a laptop to capture the dataplane packets and then try to compare them without synchronized timestamps. This one little feature would have saved us a lot of headaches !
When it comes to emerging technologies, this space between the development and actual production use - between developers and the network engineers in the trench - is something I find incredibly interesting. It's great to be involved in the development at the point you can provide substantive feedback into the actual product or technology.
And that is where we are today with Openflow. We have Openflow deployed to 4 "production" switches and have a wireless SSID in 3 buildings across campus that feeds into an Openflow switch. The cool thing is that it all pretty much works. The problem is that, when it doesn't work, it's a pretty big pain to figure out why. Yesterday I compared it to the early days of the GSRs when the tables on one of the linecards would get out of sync, but it's a bit worse because the "linecards" are spread across the whole campus and there are very few debugging tools available.
There are a number of debugging features that would be useful, but I think the most useful one would be a way to see the dataplane and control-plane packets at the same time. One way to do this would be for the switch vendors to allow you to add Openflow control-plane packets into a port-mirroring configuration. This would allow me to hook a sniffer up to a switch port and mirror both the traffic to/from a switch port and the Openflow control messages to the sniffer.
Why would this be useful ? One problem we're having right now is that some laptops take 1-2 minutes to get a DHCP lease on the Openflow network. Is the switch taking a long time to encapsulate the first DHCP message into an Openflow message and send it to the controller ? Is the controller taking a long time to send the packet-out and flow-add messages to the switch ? Are the Openflow messages getting lost along the way ? Today I have to run a tcpdump on the Openflow controller to capture control-plane packets and Wireshark on a laptop to capture the dataplane packets and then try to compare them without synchronized timestamps. This one little feature would have saved us a lot of headaches !
Thursday, May 27, 2010
Grab bag
A little trivia about myself. When I was much younger, I taught myself how to juggle for a talent show. The grand finale was to juggle 3 apples while taking a bite out of one of them each time it came around. Well, right now I feel a bit like a juggler with a dozen apples in the air, desperately trying not to drop one, so here are some snippets of what I'm up to...
- Openflow. We have our "production" Openflow network up which includes a switch in the Comm Services building, two in Wrubel and one that supports an Openflow SSID that is deployed in Lindley Hall (Computer Science), Informatics and the Innovation Center. We need to get more "testers" moved onto the switches and SSID to stress test the system. My laptop and IP phone have been on the Openflow network for almost 6 weeks without any problems. We're hoping to have a Informatics grad student working on the project with us starting in September.
- Wireless. When HP bought 3COM/H3C this spring they got what appears to be a very good controller-based wireless product from H3C. We received some eval equipment yesterday which we'll be testing. We're trying to determine the right path moving forward between HP's two different wireless systems.
- GlobalNOC Summer of Networking. Students have started arriving with the remaining students starting on June 1st. Our weekly training program will start the following week. I think we have 8 total students. Most of them are in the syseng area, but we also have one in the Service Desk and one in my area (Network Architecture), to help with the test lab.
- Test Lab. Use of the testlab is really picking up. We have a bunch of new equipment coming in on eval as well as some permanent equipment and a lot of people who want to do testing. Ed Furia and I have been supporting this in our spare time (with help from an intern this summer), but we really need to get a full-time lab admin hired.
- Training. At the GlobalNOC retreat last week there was a lot of interest in developing a training program. We're hoping to develop a curriculum of hands-on network training that could benefit GlobalNOC staff, other UITS staff, interns, and potentially others.
- IU Health Science Network: This is a design I proposed in 2007 to resolve many of the issues surrounding the IU School of Medicine and the Clarian hospital system. It's finally gaining momentum and implementation will start very soon.
Well, those are the highlights !
- Openflow. We have our "production" Openflow network up which includes a switch in the Comm Services building, two in Wrubel and one that supports an Openflow SSID that is deployed in Lindley Hall (Computer Science), Informatics and the Innovation Center. We need to get more "testers" moved onto the switches and SSID to stress test the system. My laptop and IP phone have been on the Openflow network for almost 6 weeks without any problems. We're hoping to have a Informatics grad student working on the project with us starting in September.
- Wireless. When HP bought 3COM/H3C this spring they got what appears to be a very good controller-based wireless product from H3C. We received some eval equipment yesterday which we'll be testing. We're trying to determine the right path moving forward between HP's two different wireless systems.
- GlobalNOC Summer of Networking. Students have started arriving with the remaining students starting on June 1st. Our weekly training program will start the following week. I think we have 8 total students. Most of them are in the syseng area, but we also have one in the Service Desk and one in my area (Network Architecture), to help with the test lab.
- Test Lab. Use of the testlab is really picking up. We have a bunch of new equipment coming in on eval as well as some permanent equipment and a lot of people who want to do testing. Ed Furia and I have been supporting this in our spare time (with help from an intern this summer), but we really need to get a full-time lab admin hired.
- Training. At the GlobalNOC retreat last week there was a lot of interest in developing a training program. We're hoping to develop a curriculum of hands-on network training that could benefit GlobalNOC staff, other UITS staff, interns, and potentially others.
- IU Health Science Network: This is a design I proposed in 2007 to resolve many of the issues surrounding the IU School of Medicine and the Clarian hospital system. It's finally gaining momentum and implementation will start very soon.
Well, those are the highlights !
Tuesday, May 4, 2010
Hotel California
Well, I'm spending another week in a hotel in California (Sunnyvale this time). Juniper yesterday, HP today, and Cyan Optics tomorrow. Cyan makes an interesting product in the packet/optical space. It's potentially a very good match for RONs and Statenets that don't need a lot of wave services (< 8 waves) and want a low-cost Ethernet services platform that is well integrated with the DWDM platform. Interestingly, Cyan has a connection to IU in that one of the Cyan founders, Mike Hatfield, is an Indiana native who is an alum of the IU Kelley School of Business.
Tuesday, April 20, 2010
Openflow Testbed Taking Shape
I started my morning with a nice walk across campus to Lindley Hall for a meeting with Rob Henderson. I have to interject that it was a great morning to walk across campus - 60 degrees and sunny ! It sure beats 10th and the Bypass !! Anyway the primary topic of the meeting was our Openflow testbed, but we wondered off on a number of different topics. We need to get people to help us test Openflow and Informatics & Computer Sciences seemed like a logical place to start. Rob is onboard and we're ready to start rolling !
Our first step will be to deploy a wireless SSID for Openflow. The SSID will function exactly like our 802.1x SSID (IU Secure) except the user traffic will be plumbed through a couple of Openflow enabled switches before it hits the first router. The key advantages are (1) we can easily deploy an Openflow SSID to thousands of APs to get a lot of users and (2) users can opt-in and out easily simply be switching SSIDs (which will be helpful if something breaks).
In parallel with the wireless deployment, we'll be deploying Openflow on production switches for wired users, first in the UITS complex at WCC and then at Informatics and CS. If all goes well, we could have Openflow enabled on 15-20 switches by the end of July along with the wireless deployment.
Thursday, April 8, 2010
NSF Campus Bridging Workshop
I spent yesterday at the IUPUI Conference Center attending an NSF "Campus Bridging" workshop. Your first response will likely be the same as everyone I've spoken to - "What the heck is that ?".
Well, this was the first one I attended, but I believe this is one track in a series of workshops to help the NSF decide how to structure it's future Cyberinfrastructure (CI) funding programs. The focus was on how to get campuses ramped up to support the data deluge generated by scientific instruments from gene sequencers to the LHC. Obviously networking is a big part of that equation, but certainly not the only part. There was a lot of discussion about data storage and indexing, meta data, federated identity and so on.
Here are a couple of good presentations that I think hit the nail on the head in terms of how we should be building campus networks to handle big data science applications...
Network Architecture for High Performance (Joe Metzger - ESNET)
The Data Intensive Network (Guy Almes - TAMU)
Incidentally, IU started building our campus networks this way in about 2003-04 and I think this is one of the reasons we've been so successful with projects like the Data Capacitor.
Well, this was the first one I attended, but I believe this is one track in a series of workshops to help the NSF decide how to structure it's future Cyberinfrastructure (CI) funding programs. The focus was on how to get campuses ramped up to support the data deluge generated by scientific instruments from gene sequencers to the LHC. Obviously networking is a big part of that equation, but certainly not the only part. There was a lot of discussion about data storage and indexing, meta data, federated identity and so on.
Here are a couple of good presentations that I think hit the nail on the head in terms of how we should be building campus networks to handle big data science applications...
Network Architecture for High Performance (Joe Metzger - ESNET)
The Data Intensive Network (Guy Almes - TAMU)
Incidentally, IU started building our campus networks this way in about 2003-04 and I think this is one of the reasons we've been so successful with projects like the Data Capacitor.
Thursday, April 1, 2010
Visit to Ball State
I spent yesterday afternoon at Ball State University in Muncie, IN. For those non-Hoosiers out there, Ball State is named after te Ball Family as in the jars you can tomatoes in ! I met with Steve Jones who is the director of their CICS program. Hopefully I don't butcher the acronym, but IIRC it stands for Center for Information and Communications Sciences. It's a very cool program and as someone who grew up right down the road from the university, I had no idea it existed. They have some very bright and motivated students and hopefully some of them will eventually come join the team at the GlobalNOC !
-- Post From My iPhone
-- Post From My iPhone
Wednesday, March 24, 2010
Announcing the GlobalNOC Summer of Networking Program
The GlobalNOC has a long history of hiring students to work on projects during the summer. In fact, many of our software developers and system administrators started with us as students. This summer we anticipate having about 8-10 students working in multiple areas of the GlobalNOC including Systems Engineering, Service Desk and Network Architecture.
With such a large group of students, we decided to pilot a program to provide additional training opportunities in a group forum. Our plan is to have group training/seminar sessions one afternoon a week. The initial sessions with include presentations and training by GlobalNOC staff to the students. Towards the end of the summer, the sessions will be focused on the students presenting their work from the summer or a networking topic they've been researching during the summer to each other. Since the students will be split between the IUPUI and IUB campuses, most of the group sessions will be conducted via high-definition video conferencing, but at least two of the sessions will be conducted face-to-face with all the students.
There will also be opportunities for students to shadow GlobalNOC staff in areas other than the area in which they are working. So a student working on software development in the Systems Engineering group would have a chance to learn about the Service Desk, Network Engineering and Network Architecture groups by shadowing someone in each of those areas.
This is a pilot, so we may need to make adjusts during the summer, but I think this will be a great opportunity for students to get hands-on experience managing large-scale networks.
With such a large group of students, we decided to pilot a program to provide additional training opportunities in a group forum. Our plan is to have group training/seminar sessions one afternoon a week. The initial sessions with include presentations and training by GlobalNOC staff to the students. Towards the end of the summer, the sessions will be focused on the students presenting their work from the summer or a networking topic they've been researching during the summer to each other. Since the students will be split between the IUPUI and IUB campuses, most of the group sessions will be conducted via high-definition video conferencing, but at least two of the sessions will be conducted face-to-face with all the students.
There will also be opportunities for students to shadow GlobalNOC staff in areas other than the area in which they are working. So a student working on software development in the Systems Engineering group would have a chance to learn about the Service Desk, Network Engineering and Network Architecture groups by shadowing someone in each of those areas.
This is a pilot, so we may need to make adjusts during the summer, but I think this will be a great opportunity for students to get hands-on experience managing large-scale networks.
Monday, March 22, 2010
Openflow Trip
Nothing like back-to-back weeks of travel ! This week we're headed to Silicon Valley for a series of meetings related to Openflow including stops at Stanford University and HP Labs. Actually, last week's trip to GEC7 at Duke University was related to Openflow as well. If you haven't checked out Openflow yet, I'd encourage you to do so (www.openflowswitch.org). It's a standard API that allows external systems (think PC servers) to manipulate the forwarding ASICs in switches and routers. IU was recently awarded an NSF grant through the GENI program to help get Openflow deployed on campuses.
Monday, March 15, 2010
Heading to GEC 7
I'm heading to GEC 7, the 7th GENI Engineering Conference, tomorrow with several colleagues from the IU GlobalNOC. IU has received multiple GENI grants so far including one for the Openflow Campus Trials which I'm working on along with the PI, Chris Small. Tomorrow night we'll be doing a demo of our current Openflow deployment that includes 6 HP switches running Openflow capable code along with the NOX and SNAC Openflow controllers. You can check out our project page on the GENI Wiki for more information.
Tuesday, January 5, 2010
Back in the Saddle Again !
Happy New Year ! Hopefully everyone enjoyed the holidays. I hardly looked at email for 2 full weeks which was very nice !
2010 promises to be as busy and eventful as 2009, if not more ! We are in the midst of two separate beta testing programs right now along with an RFP. I'm actively working on two grant proposals, a major project to provide a more seamless networking experience across the Clarian (hospital) and IU facilities, and I'm trying to finish up a Legacy RSA with ARIN. In all, my group has about 20 active projects on our plate right now !
2010 promises to be as busy and eventful as 2009, if not more ! We are in the midst of two separate beta testing programs right now along with an RFP. I'm actively working on two grant proposals, a major project to provide a more seamless networking experience across the Clarian (hospital) and IU facilities, and I'm trying to finish up a Legacy RSA with ARIN. In all, my group has about 20 active projects on our plate right now !
Subscribe to:
Posts (Atom)
Posts
