Joint Techs Workshop Albuquerque, NM 2/7/2006 Some thoughts on deploying measurement infrastructures Dan Magorian Director of Engineering and Operations Mid-Atlantic Crossroads

Joint Techs Workshop Albuquerque, NM 2/7/2006 MAX recently hosted a Perf workshop Went well, I was happy about participation Until I counted institutions represented instead of people. Only 4 out of 40, inc MAX folks. Started thinking: why do some people and institutions “get” the need for deploying a measurement infrastructure, and others not? Not really cost or effort: just a couple pcs and a day Realized that people who came and got around to actually doing it, were probably people who had been burned by lack of it at least once. Everyone else, on their list for things they should be doing, but never quite rises to the top.

Joint Techs Workshop Albuquerque, NM 2/7/2006 We talked about this at Gigapop Geeks BOF Folks had some interesting comments: “We know places that get burned over and over again, yet don’t get around to deploying them”. Given how easy it is to do, this is depressing. “The problem is that they’re weird kernals that are different from our usual service boxes. If they looked like our standard campus distro, we’d have done a lot more of them.” Effort making sure they’re secured, patched, etc. “The retention rate of configured boxes after workshops that remain deployed is only around 25%” After people put the effort into configuring them, why?

Joint Techs Workshop Albuquerque, NM 2/7/2006 So to try and help this situation At MAX, we’re completing a pair of portable mini Spacewalker-style pcs that have onboard giges and 10G Myricom pci-express nics that take XFPs, with iperf tuned and ready. We needed them ourselves for testing 10G lambdas that we bring up for customers. But as a side benefit, they will be loaners for MAX participants experiencing performance problems, to borrow for few days for “divide-and-conquer” testing. We’re also considering MAX-supplied pcs that go on each participant edge router. Like old AMP pcs, easier if people just have to plug them in. But don’t get “buy in”

Joint Techs Workshop Albuquerque, NM 2/7/2006 Next topic: Best places to put them. For Abilene, ESnet, national backbones: One or more at all router nodes. Gigapops: At least one on routers facing national backbones, better one on each router node. But think about non-congruent L1/L2/L3 infrastructures these days. Fewer with router consolidation? MAX used to have mostly a router at each node, but these days uses optical backhaul from 6 pops to “big fat routers” (Juniper T640s) in middle. Putting a L3 iperfer on a remote L2 switch may not accomplish much more than testing one of many lambdas. Points out lack of tools at L1/L2.

Joint Techs Workshop Albuquerque, NM 2/7/2006 Best places to put boxes on campuses The minimum is easy: one iperfer on each edge router facing national backbones. Other test boxes can be centralized If everyone even did that much, would be really good More starts to get complicated. How do you pick sample “strategic locations”? At compute clusters and critical resources are obvious choices. Behind firewalls/nat boxes? “Worst-case” subnets? One per campus switch? Might be 100s. How many is affordable or reasonable? Diminishing returns?

Joint Techs Workshop Albuquerque, NM 2/7/2006 Once boxes deployed, getting maximum benefit If just lie around doing testing on demand, may not establish “everything working” baseline needed for problem resolution. MAX had iperfers since 2000, but only now starting to schedule regular testing with bwctl. Coordinating gigapop-campus, inter-gigapop, and gigapop-Abilene testing takes more people effort. Benefits clear, but not many get around to doing it. Looking forward to new perfSonar framework still under development and benefits that will bring with mode autodiscovery mechanisms.