Building a Community Cyberinfrastructure to Support Marine Microbial Ecology Metagenomics Invited Talk Metagenomics 2006 UCSD La Jolla, CA October 4, 2006 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD

Challenge: Average Throughput of NASA Data Products to End User is < 50 Mbps Tested October Internet2 Backbone is 10,000 Mbps! Throughput is < 0.5% to End User

Dedicated Optical Channels Makes High Performance Cyberinfrastructure Possible (WDM) Source: Steve Wallach, Chiaro Networks “Lambdas” Parallel Lambdas are Driving Optical Networking The Way Parallel Processors Drove 1990s Computing

San Francisco Pittsburgh Cleveland National Lambda Rail (NLR) and TeraGrid Provides Cyberinfrastructure Backbone for U.S. Researchers San Diego Los Angeles Portland Seattle Pensacola Baton Rouge Houston San Antonio Las Cruces / El Paso Phoenix New York City Washington, DC Raleigh Jacksonville Dallas Tulsa Atlanta Kansas City Denver Ogden/ Salt Lake City Boise Albuquerque UC-TeraGrid UIC/NW-Starlight Chicago International Collaborators NLR 4 x 10Gb Lambdas Initially Capable of 40 x 10Gb wavelengths at Buildout NSF’s TeraGrid Has 4 x 10Gb Lambda Backbone Links Two Dozen State and Regional Optical Networks DOE, NSF, & NASA Using NLR

The OptIPuter Project – Creating High Resolution Portals Over Dedicated Optical Channels to Global Science Data NSF Large Information Technology Research Proposal –Calit2 (UCSD, UCI) and UIC Lead Campuses—Larry Smarr PI –Partnering Campuses: SDSC, USC, SDSU, NCSA, NW, TA&M, UvA, SARA, NASA Goddard, KISTI, AIST, CRC(Canada), CICESE (Mexico) Engaged Industrial Partners: –IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent $13.5 Million Over Five Years—Now In the Fifth Year NIH Biomedical Informatics Research Network NSF EarthScope and ORION

OptIPuter Software Architecture--a Service-Oriented Architecture Integrating Lambdas Into the Grid GTPXCPUDT LambdaStream CEPRBUDP DVC Configuration Distributed Virtual Computer (DVC) API DVC Runtime Library Globus XIO GRAM GSI Distributed Applications/ Web Services Telescience Vol-a-Tile SAGEJuxtaView Visualization Data Services LambdaRAM DVC Services DVC Core Services DVC Job Scheduling DVC Communication Resource Identify/Acquire Namespace Management Security Management High Speed Communication Storage Services IP Lambdas Discovery and Control PIN/PDC RobuStore Source: Andrew Chien, UCSD

Calit2 “Lives in the Future” By Building Systems of Emerging Disruptive Technologies Co-Evolution of Personal Automobile and Highway/Petroleum Infrastructure Source: Harry Dent, The Great Boom Ahead Calit2 Works Here { Technologies Diffuse Into Society Following an S-Curve

Calit2--A Systems Approach to the Future of the Internet and its Transformation of Our Society Calit2 Has Assembled a Complex Social Network of Over 350 UC San Diego & UC Irvine Faculty Working in Multidisciplinary Teams With Staff, Students, Industry, and the Community Integrating Technology Consumers and Producers Into “Living Laboratories”

Calit2 Brings Computer Scientists and Engineers Together with Biomedical Researchers Some Areas of Concentration: –Metagenomics –Genomic Analysis of Organisms –Evolution of Genomes –Cancer Genomics –Human Genomic Variation and Disease –Proteomics –Mitochondrial Evolution –Computational Biology –Information Theory and Biological Systems UC San Diego UC Irvine

Evolution is the Principle of Biological Systems: Most of Evolutionary Time Was in the Microbial World You Are Here Source: Carl Woese, et al Much of Genome Work Has Occurred in Animals

PI Larry Smarr Paul Gilna Ex. Dir. Calit2 is Now Attracting Private Foundation Grants Announced January 17, $24.5M Over Seven Years

Marine Genome Sequencing Project – Measuring the Genetic Diversity of Ocean Microbes Sorcerer II Data Will Double Number of Proteins in GenBank!

Current Universe of Medium/ Large Protein Families Source: Shibu Yooseph, et al. (PLOS Biology in press 2006) Protein Families Conserved Across Tree of Life Protein Families Unique to GOS 17,067 Protein Family Clusters

Flat File Server Farm W E B PORTAL Traditional User Response Request Dedicated Compute Farm (1000 CPUs) TeraGrid: Cyberinfrastructure Backplane (scheduled activities, e.g. all by all comparison) (10000s of CPUs) Data- Base Farm 10 GigE Fabric Calit2’s Direct Access Core Architecture Will Create Next Generation Metagenomics Server Source: Phil Papadopoulos, SDSC, Calit2 + Web Services Sargasso Sea Data Sorcerer II Expedition (GOS) JGI Community Sequencing Project Moore Marine Microbial Project NASA Goddard Satellite Data Community Microbial Metagenomics Data Web (other service) Local Cluster Local Environment Direct Access Lambda Cnxns

The Bioinformatics Core of the Joint Center for Structural Genomics will be Housed in the Building Extremely Thermostable -- Useful for Many Industrial Processes (e.g. Chemical and Food) 173 Structures (122 from JCSG) Determining the Protein Structures of the Thermotoga Maritima Genome 122 T.M. Structures Solved by JCSG (75 Unique In The PDB) Direct Structural Coverage of 25% of the Expressed Soluble Proteins Probably Represents the Highest Structural Coverage of Any Organism Source: John Wooley, UCSD

Interactive Visualization of Thermatoga Proteins at Calit2 Source: John Wooley, Jurgen Schulze, Calit2

OptIPortal– Termination Device for the OptIPuter Global Backplane 20 Dual CPU Nodes, 20 24” Monitors, ~$50,000 1/4 Teraflop, 5 Terabyte Storage, 45 Mega Pixels--Nice PC! Scalable Adaptive Graphics Environment ( SAGE) Jason Leigh, EVL-UIC Source: Phil Papadopoulos SDSC, Calit2

NW! CICESE UW JCVI MIT SIO UCSD SDSU UIC EVL UCI OptIPortals OptIPortal Calit2 is Now OptIPuter Connecting Remote Moore-Funded Microbial Researchers

Calit2 and the Venter Institute Will Combine Telepresence with Remote Interactive Analysis OptIPuter Visualized Data HDTV Over Lambda Live Demonstration of 21st Century National-Scale Team Science 25 Miles Venter Institute

Created in Reykjavik, Iceland 2003 Countries are Aggressively Creating Gigabit Services: Interactive Access to CAMERA and LOOKING Systems Visualization courtesy of Bob Patterson, NCSA.

New OptIPuter Driver: Gigabit Fibers on the Ocean Floor -- Controlling Sensors and HDTV Cameras Remotely National Science Foundation Is Planning a New Generation of Ocean Observatories –Ocean Research Interactive Observatory Networks (ORION) Fibered Observatories Linked to Land Fiber Infrastructure Laboratory for the Ocean Observatory Knowledge Integration Grid (LOOKING) –Building a Prototype Based on OptIPuter Technologies Plus Web/Grid Services –HDTV Streams Over IP Will be a Major Driver (Funded by NSF ITR- John Delaney, UWash, PI) LOOKING is Driven By NEPTUNE CI Requirements Making Management of Gigabit Flows Routine

Using the OptIPuter to Couple Data Assimilation Models to Remote Data Sources Including Biology Regional Ocean Modeling System (ROMS) NASA MODIS Mean Primary Productivity for April 2001 in California Current System

Deploying Novel Infrastructure Enables New Science: Gigabit Fibers on the Ocean Floor Source: John Delaney & Deborah Kelley, UWash Canadian-U.S. Collaboration An Experiment in the NSF Laboratory for the Ocean Observatory Knowledge Integration Grid (LOOKING) ITR Prototype of CI for NSF’s ORION

High Definition Still Frame of Hydrothermal Vent Ecology 2.3 Km Deep White Filamentous Bacteria on 'Pill Bug' Outer Carapace 1 cm. Source: John Delaney and Research Channel, U Washington

A Near Future Metagenomics Fiber Optic-Enabled Data Generator Source John Delaney, UWash