Christopher Paolini Computational Science Research Center San Diego State University 100G and Beyond Workshop: Ultra High Performance Networking in California Calit 2 Auditorium First floor, Atkinson Hall UC San Diego La Jolla, CA Tuesday, February 26, 2013 · Campus and Lab Strategies Panel · 11:00AM – 12:00PM

 University network operations centers support multiple, conflicting missions.  Network Security or Network Performance: which is more important? vs.  NOCs typically accountable to university business divisions and contend with legal and public relations pressures → security wins always.  NOCs not usually accountable to research groups (often never communicate with faculty).  University enterprise (e.g. general purpose/financial/personal) computing: security > performance  Computational and “Big Data” research: performance > security  What can we do to ensure efficient scientific data transfer between universities and national labs?

 A network optimized for business is not designed or capable of supporting data intensive science.  Universities will always need to support security features that protect organizational financial and personnel data.  Solution: create separate data intensive science network, external to university enterprise network  Design formalized by ESnet, based on traditional network DMZ paradigm

 Science DMZ: (1) dedicated access to high-performance WAN, (2) high-performance switching infrastructure (large buffer memory), (3) dedicated data transfer nodes

 Science DMZ using CENIC California Research and Education Network resources

Alcatel-Lucent 10 and 40 Gbps switching devices, per CSU policy DMZ spans four campus buildings: Administration, Life Sciences (CSRC Data Center), Education & Business Administration (UCO Data Center), and Chemical Sciences (VizCenter) Primary users: CSRC affiliated faculty and students AL OmniVista 2500 for network management NSF Office of CyberInfrastructure CC-NIE Grant

Computational science network connects to the DMZ Funded in 2009 through NSF MRI award Cisco 10 Gbps Catalyst 4900M switching devices CSRCnet spans five campus buildings: Administration, Life Sciences (CSRC Data Center), Education & Business Administration (UCO Data Center), Physics, and Engineering Sole users: CSRC affiliated faculty and students 10G access to SDSC

 Facilitate high-performance data transfer for scientific applications using Globus Online GridFTP  Alcatel-Lucent OmniSwitch 10K (core device)  Two Alcatel-Lucent OmniSwitch 6900s (satellite devices)  Dedicated and independent 10GE (maybe 40GE) uplink to Internet2 and ESnet via CENIC  Optimized network for high-volume bulk transfer of scientific datasets  Unencumbered, high-speed access to online scientific applications and data generated at SDSU  External access to science resources not impacted by regular “enterprise” or business class Internet traffic  Focus on “BigData” Intensive Science: earthquake rupture and wave propagation, parallel 3D unified curvilinear coastal ocean modeling, geologic sequestration simulation of supercritical CO 2, large-scale proteomic data, bioinformatics of gene promoter analysis, microbial metagenomics, and high-order PSIC methods for simulation of pulse detonation engines  Network performance measurement based on the PerfSONAR framework  InCommon Federation global federated system for identity management and authentication to DMZ connected hosts and services

 Extension of the standard, two channel FTP protocol  Control Channel ◦ Command/Response ◦ Used to establish data channels ◦ Basic file system operations (e.g. mkdir, delete, etc.)  Data channel: Pathway over which file is transferred  Scheduled transfers using command line interface: $ scp xsede#lonestar4:~/GO/bigdatafile xsede#trestles:~/GO/bigdatafile $ scp xsede#trestles:~/GO/bigdatafile paolini#sdsu:~/GO/bigdatafile

 Science DMZ performance monitoring accomplished using perfSONAR tool suite  Server side tools run on designated hosts attached to key switches  End-to-end testing with collaborating perfSONAR sites  Determine one way latencies and packet loss between hosts using One-Way Active Measurement Protocol (OWAMP) owping -c i.01 remotedmz  Periodic throughput tests to remote Science DMZs using Bandwidth Test Controller (BWCTL)  Resource allocation and scheduling daemon for regularly- scheduled Iperf tests bwctl -s remotedmz -P 4 -t 30 -f M -w 4M -S 32

 U.S. education and research identity federation service  Provides common framework for trusted shared management of access to on-line resources  Provide users single sign-on convenience and privacy protection – Shibboleth Service Provider Federating software  Site admins can delegate responsibility for administering service provider (SP) metadata to another admin

 Primary SDSU faculty/staff for Science DMZ implementation: NameRole Phone Christopher PaoliniCSRC Affiliated Faculty, Network Engineering and Research Jose CastilloDirector of Computational Science Research Center Rich PickettCampus Kent McKelveyDirector of Network Skip AustinNetwork Planning and Gene LeDucTechnology Security Officer Robert OsbornInfrastructure Installation, Configuration, and Support  Current and planned DMZ related research: Development of new transport layer protocols that use compressed sensing techniques to perform sparse sampling on streaming petabyte sized datasets originating from remote CO 2 sequestration, curvilinear coastal ocean modeling, and earthquake rupture and wave propagation simulations Development of a new Alcatel-Lucent SDN/Application Fluent Network based protocol for the OS10K that bridges Lustre RDMA traffic between 40GE and FDR InfiniBand