Office of Science U.S. Department of Energy ESCC Meeting July 21-23, 2004 Network Research Program Update Thomas D. Ndousse Program Manager Mathematical, Informational, and Computational Sciences (MICS Division)

Office of Science U.S. Department of Energy Program Goals What's new New SciDAC and MICS Network Research Projects 1.Ultra-Science Network Testbed – Base funding 2.ESnet MPLS Testbed – Base funding 3.Application Pilot Projects (Fusion Energy, Biology, Physics) 4.GMPLS Control Plane 5.GridFTP Lite (Generalized File Transfer Protocol) 6.Transport protocols for switched dedicated links 7.Cyber security: IDS and group security 8.Data grid wide area network monitoring for LHC Gap (Network-enabled storage systems) Leadership Class National Supercomputer Budget Reduction in FY-04 & FY-05 Budget SC Network PI meeting in late September, 2004

Office of Science U.S. Department of Energy Revised Program Focus Previous Focus R&D on fundamental networking issues Single and small group of investigators Limited emphasis on technology transfer and integration Limited emphasis on network, middleware, and applications integration New Focus Applied research, engineering, and testing Experimental networking using UltraNet and MPLS testbed Integrated applications, middleware, an networks prototypes Leadership-class supercomputing –Impact on network research –Impact on research testbeds –Impact on inter-agency network coordination activities

Office of Science U.S. Department of Energy Network Research Program Elements Program Elements  R&D, E: Research, Development and Engineering  ANRT: Advanced Network Research Testbeds (ANRT)  ECPI: Early Career Principal Investigators  SBIR: Small Business innovation Research

Office of Science U.S. Department of Energy FY-03/04 Network Research Program Budget YearFY-03/04 SciDAC$2.0M MICS$4.5M Total$6.5M YearFY-04/5 SciDAC$2.0M MICS$2.5M Total$4.5M

Office of Science U.S. Department of Energy Implementation of Office of Science Networking Recommendations – I (Very High-Speed Data Transfers) Data, data, data, data everywhere! Many science areas such high energy physics, computational biology, climate modeling, astro-physics, etc., predict a need for multi-Gbits/sec data transfer capabilities in the next 2 years Program Activities Scalable TCP protocol enhancements for share networks Scalable UDP for share networks and dedicated circuits Alternative TCP/UDP transport Protocols Bandwidth on-demand technologies GridFTP lite Ultra high-speed network components

Office of Science U.S. Department of Energy Implementation of Office of Science Networking Recommendations – II (Diverse SC Network Requirements) Optical Layer Hybrid-Switched Links Circuit-Switched Links Packet-Switched Links Logical Network Layer UDP Variants Others TCP Variants TCP High-Performance Middleware High-End Science Applications Control and Signaling Plane Problem Many science areas such high energy physics, computational biology, climate modeling, astro-physics, etc., predict a need for multi-Gbits/sec data transfer capabilities in the next 2 years Program Vision

Office of Science U.S. Department of Energy Implementation of Office of Science Networking Recommendations - III Production Networks  Connects all DOE sites  7x24 and highly reliable  Advanced Internet capability  Predominantly best-effort Advanced Research Network  Experimental optical inter-networking  On-demand bandwidth/DWDM circuits  Ultra high protocol development/testing  GMPLS High-Impact Science Network Connect few high-impact science sites Ultra high-speed IP network technologies Reliable and secure services QoS/MPLS for on-demand bandwidths Ultra-Science Network, 20 Gbps ESnet QoS/MPLS Testbed, 5 Sites ESnet

Office of Science U.S. Department of Energy Impact of MPLS and Ultra-Science Networks Testbeds Category A Sites- Sites w/local fiber arrangements 1.FNAL OC-12/OC Tier 1 - CMS 2.ANLOC-12/OC ORNLOC-12/OC Leadership Computing 4.PNNLOC-12/OC EMSL Computing 5.NERCSOC-48/OC Flagship Computing 6.LBLOC-48/OC SLACOC-12/OC BABAR Data Source Category B Sites- Sites w/local fiber arrangements (T3 to OC-12) 1.BNL --- Tier 1 - ATLAS 2.JLAB 3.GA 4.Princeton 5.MIT 1.Use UltraNet to link site with local fiber connectivity 2.Develop dynamic provisioning technologies to manage DWDM circuits 3.Develop and test advanced transport protocols for high-speed data transfers over DWDM links 1.Use MPLS to establish LSPs to link sites with high- impact applications 2.Use MPLS to provide guaranteed end-to-end QoS to high-impact applications 3.Link LSPs with dynamics GMPLS circuits established

Office of Science U.S. Department of Energy Advanced Research Network Testbeds: (QoS+MPLS) Technical Activities Deployment of site QoS technologies at selected DOE sites Integrate QoS with local grid infrastructure Deployment of MPLS in ESnet core network Integrate MPLS integration with GMPLS Integrate on-demand bandwidth technologies to application Target Science Applications High Energy (CMS & ATLAS) – High-speed data transfer Fusion Energy - remote control of scientific instruments Nuclear Physics – remote collaborative visualization Goal To develop advanced network technologies to provide guaranteed on-demand end-to-end bandwidth to selected high-impact science applications

Office of Science U.S. Department of Energy Initial MPLS Deployment in ESnet GA BNL Starlight CERN JLab PNNL NERCS Caltech SLAC Starlight ORNL QoS/MPLS GMPLS GMPLS Site QoS MPLS Site Technology:QoS Core technologies: MPLS Core Technologies:MPLS & GMPLS FNAL

Office of Science U.S. Department of Energy Ultra-Science Network Testbed: Topology Upgrade: 20 Gbps backbone DOE University Partners DOE National Lab CERN Sunnyvale FNAL CalTech SLAC LBL NERSC PNNL 10 Gbps ESnet Links 10 Gbps UltraNet Link under discussion Major Nodes StarLight/FNAL SOX/ORNL Seattle/PNNL Sunnyvale/SLAC Sunnyvale/Caltech StarLight ORNL CalTech BNL JLab ESnet 10 Gbps 20 Gbps SOX

Office of Science U.S. Department of Energy Dynamic Provisioning Development data circuit-switched technologies IP control plane based on GMPLS Integration of QoS, MPLS, and GMPLS Inter-domain control plane signaling Bandwidth on-demand technologies Ultra High-Speed Data Transfer Protocols High-speed transport protocols for dedicated channels High-speed data transfer protocols for dedicated channels Layer data multicasting Ultra High-Speed Cyber Security Ultra high-speed IDS Ultra high-speed firewalls and alternatives Control plane security Ultra-Science Network Testbed: Activities

Office of Science U.S. Department of Energy UltraNet/GMPLS Institutions FNALFiber Starlight/UltraNet ORNLFiber to Atlanta and Starlight/UltraNet SLACFiber to Sunnyvale/UltraNet (under discussion) PNNLFiber connection to Seattle/UltraNet CaltechDWDM link to Sunnyvale/UltraNet UltraNet QoS/MPLS Fusion Energy:GA, NERCS, Princeton ATALS Project:BNL, CERN, U. Michigan CMS Project: FNAL, CERN, UCSD Funded Projects: Application Development FANLExplore very high-speed transfer of LHC data on UltraNet PNNLRemote visualization of computational biology on UltraNet ORNLAstrophysics real-time data visualization on UltraNet & CHEETAH G AWide Area Network QoS using MPLS BNLExploring QoS/MPLS for LHC data transfers UltraNet funded Projects and Laboratory initiatives

Office of Science U.S. Department of Energy Inter-Agency Collaboration CHEETAHNSF:Dynamic Provisioning – Control plane interoperability Application - Astrophysics (TSI) DRAGONNSF:Dynamic Provisioning – Control plane interoperability All-optical network technology OMNINetNSF:Dynamic Provisioning – Control plane interoperability All-optical network technology UltraNetDOE:Dynamic Provisioning – Control plane interoperability Hybrid Circuit/packet switched network HOPIInternet2- Collaborations Collaboration Issues Control plane architecture and interoperability Optical service definitions and taxonomy Inter-domain circuit exchange services GMPLS and MPLS (ESnet & Internet2) integration Testing of circuit-based transport protocols Integration of network-intensive applications Integration with Grid applications

Office of Science U.S. Department of Energy UltraNet Operations and Management Engineering Team 1.UltraNet Engineering 2.ESnet Engineering representatives 3.Application Developers representatives Research Team – Awards Pending 1.Network Research PIs 2.Application Prototyping PIs 3.Other Research Networks Management Team UltraNet Engineering ESnet Engineering Rep ESCC Rep Management Responsibilities * 1.Prioritize experiments on UltraNet 2.Schedule testing 3.Develop technology transfer strategies * Needs to be integrated into the Office of Science networking governance model articulated in the roadmap workshop

Office of Science U.S. Department of Energy Network Technologies for Leadership Class Supercomputing Leadership super computer being built at ORNL National resource Access from university, national labs, and industry is a major challenge Impact of leadership class supercomputer on Office of science networking Network technologies for leadership class supercomputer Inter-agency networking coordination issues

Office of Science U.S. Department of Energy Network Technologies for Leadership Class Supercomputing Leadership super computer being built at ORNL National resource Access from university, national labs, and industry is a major challenge Impact of leadership class supercomputer on Office of science networking Network technologies for leadership class supercomputer Inter-agency networking coordination issues

Office of Science U.S. Department of Energy Computing and Communications: The “impedance” mismatch: computation and communication Rule of thumb: The bandwidth must be adequate to transfer Petabyte/day ~ 200Gbps - NOT on the evolutionary path of backbone, much less application throughput 80 Gbps 100 Gbps Cray 1: 133M T1 Earth Simulator 37T Cray Y-MP:400M Intel Paragon 150G ASCI Blue Mountain: 3T ASCI White: 12T 10 Gbps SONET 40 Gbps SONET 1.E+02: 100M 1.0E+03: 1G 1.0E E+05 1.E+06: 1T 1.E+07: 10T 1.E+08: 100T 1.E+09:1P 10 Gbps 40 Gbps 1 Gbps 2.5 Gbps SONET 0.6 Gbps SONET 0.15 Gbps SONET T3 10 Mbps Ethernet 100 Mbps Ethernet 1 GigE Ethernet Supercomputer peak performance Backbone performance Achievable end-to-end performance by applications projected

Office of Science U.S. Department of Energy Q&A