Scaling-Up the BIRN Building on the BIRN Workshop National Institutes of Health Bethesda, MD March 22, 2004 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technologies Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD

BIRN is a BISTI Leadership Project for the National Institutes of Health Biomedical Information Science and Technology Initiative –BISTI Report to NIH Director: June 1999 Interdisciplinary Teams –Collaborating to Develop Mathematical Models, Write Software and Adapt Systems Cooperation with Computer Science in: –Algorithm, Software, Database and System Development Improved Access to Computing –Shared PC Clusters in Biomedical Laboratories to High-Performance Systems in National Centers Workforce Development to Encompasses Every Level –Technician to Ph.D. New Centers for Research and Teaching –People Competent in Biomedicine and Computation & Mathematics Botstein-Smarr Report June 1999

Cal-(IT) 2 --An Integrated Approach the Future of the Internet UC San Diego & UC Irvine Faculty Working in Multidisciplinary Teams With Students, Industry, and the Community BIRN Driven

BIRN Cyberinfrastrucure Must Evolve Riding Multiple Exponentials Scientific American, January 2001 Number of Years Performance per Dollar Spent Data Storage (bits per square inch) (Doubling time 12 Months) Optical Fiber (bits per second) (Doubling time 9 Months) Silicon Computer Chips (Number of Transistors) (Doubling time 18 Months)

Data Intensive e-Science Will Require Common Cross-Agency Cyberinfrastructure

The OptIPuter Project – Removing Bandwidth as an Obstacle In Data Intensive Sciences NSF Large Information Technology Research Proposal –Cal-(IT) 2 and UIC Lead CampusesLarry Smarr PI –Co-PIs Mark Ellisman, Phil Papadopoulos, Tom DeFanti, Jason Leigh –USC, SDSU, NW, Texas A&M, Univ. Amsterdam Partnering Campuses Industrial Partners –IBM, Sun, Telcordia/SAIC, Chiaro Networks, Calient, Glimmerglass $13.5 Million Over Five Years Optical IP Streams From Lab Clusters to Large Data Objects NIH Biomedical Informatics NSF EarthScope and ORION siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml Research Network

What is the OptIPuter? Next Generation Cyberinfrastructure Applications Drivers Interactive Large Data Objects OptIPuter Nodes Scalable PC Cluster LambdaGrid Browser IP over Lambda Connectivity Predictable Network Open Source LambdaGrid Middleware Can Reserve Network Data Retrieval and Mining Global Virtual Disk Drives High Def Visualization, Collaboration UltraReality TV

See Nov 2003 CACM For Articles on OptIPuter Technologies BIRN

OptIPuter Driver: On-Line Microscopes Creating Very Large Biological Montage Images 2-Photon Laser Confocal Microscope –High Speed On-line Capability Montage Image Sizes Exceed 16x Highest Resolution Monitors –~150 Million Pixels! Use Graphics Cluster with Multiple GigEs to Drive Tiled Displays Source: David Lee, NCMIR, UCSD IBM 9M Pixels

Ultra-Resolution Displays Driven by Graphics Clusters Scaling to 100 Million Pixels JuxtaView (UIC EVL) on Tiled LCD GeoWall2 –Digital Montage Viewer –8000x3600 Pixel Resolution~30M Pixels Now –Building a 75M Pixel Wall Display Is Powered By: –16 PCs with Graphics Cards –2 Gigabit Networking per PC Linked by Dedicated Multi- Gigabit Light Pipes –Research on Adding Stereo HD Collaboration Source: Jason Leigh, EVL, UIC; USGS EROS Brain Microscopy

Exploring High Performance LambdaGrid Collaboration Falko Kuester, Cal-(IT) Collaboration Laboratory with Smart Boards and Optically Connected Large Screens With UCI Potkin and UCSD Ellisman BIRN Laboratories

Cyberinfrastructure in Design Phase-- Fiber Optic Cables Satellite Wireless OptIPuter Driver: Gigabit Fibers on the Ocean Floor NSFs ORION-Ocean Research Interactive Ocean Network Based on BIRN Cyberinfrastructure

½ Mile SIO SDSC CRCA Phys. Sci - Keck SOM JSOE Preuss 6 th College SDSC Annex Node M Earth Sciences SDSC Medicine Engineering High School To CENIC Collocation Source: Phil Papadopoulos, SDSC; Greg Hidley, Cal-(IT) 2 The UCSD OptIPuter Deployment UCSD is Prototyping a Campus-Scale OptIPuter SDSC Annex Juniper T Tbps Backplane Bandwidth 20X Chiaro Estara 6.4 Tbps Backplane Bandwidth Dedicated Fibers Between Sites Link Linux Clusters NCMIR and BIRN CC

14 The 30-Year Old TCP/IP Model of Sharing the Internet Has Its Penalties Shared Internet2 Speed Limit ~50 Mbps On a 10,000 Mbps Backbone! (½ of 1% Efficiency) Similar Results for BIRN

The OptIPuter Will Become a National-Scale Collaboratory in 2004 Source: Tom West, CEO, NLR Chicago OptIPuter StarLight NU, UIC SoCal OptIPuter USC, UCI UCSD, SDSU National Lambda Rail Partnership Serves Very High-End Experimental and Research Applications 4 x 10Gb Wavelengths Initially Capable of 40 x 10Gb wavelengths at Buildout NASA Ames NASA Goddard NEPTUNE In Discussion Paves the Way for BIRN II

OptIPuter Infrastructure Is International: Global Lambda Integrated Facility: GLIF DWDM SURFnet 10 Gbit/s SURFnet 10 Gbit/s SURFnet 10 Gbit/s IEEAF 10 Gbit/s Dwingeloo ASTRON/JIVE Dwingeloo ASTRON/JIVE Prague CzechLight Prague CzechLight 2.5 Gbit/s NSF 10 Gbit/s Stockholm NorthernLight Stockholm NorthernLight CA*net4 2.5 Gbit/s New York MANLAN New York MANLAN Tokyo WIDE Tokyo WIDE 10 Gbit/s IEEAF 10 Gbit/s 2.5 Gbit/s Tokyo APAN Tokyo APAN Amsterdam NetherLight Amsterdam NetherLight Geneva CERN Geneva CERN London UKLight London UKLight Chicago StarLight Chicago StarLight Source: Kees Neggers, SURFnet