High Performance Cyberinfrastructure Discovery Tools for Data Intensive Research Larry Smarr Prof. Computer Science and Engineering Director, Calit2 (UC.

Slides:

Advertisements

Similar presentations

High Performance Cyberinfrastructure Is Required for the Era of Big Data Opening Workshop Presentation Whither Science in Mexico: an Analysis for Action.

Advertisements

Three Disruptive Leadership Opportunities for Washington State to Live in the Future Keynote Talk Washington Innovation Summit: New Decade, New Partnerships,

Calit2 " Talk Nortel Visiting Team December 12, 2005 Dr. Larry Smarr Director, California Institute for Telecommunications and Information.

Calit2: Experiments in Living in the Virtual/Physical World Pat Ledden Memorial Luncheon Faculty Luncheon Seminar UC San Diego Faculty Club February 24,

Calit2: a SoCal UC Infrastructure for Innovation Welcoming Talk Visit to Calit2 by The Trusteeship April 24, 2010 Dr. Larry Smarr Director, California.

Advancing the Metagenomics Revolution Invited Talk Symposium #1816, Managing the Exaflood: Enhancing the Value of Networked Data for Science and Society.

The Coming Revolution in Environmental Awareness

Coupling Australias Researchers to the Global Innovation Economy Fourth Lecture in the Australian American Leadership Dialogue Scholar Tour Swinburne University.

Health Sciences Driving UCSD Research Cyberinfrastructure Invited Talk UCSD Health Sciences Faculty Council UC San Diego April 3, 2012 Dr. Larry Smarr.

High Performance Cyberinfrastructure Enabling Data-Driven Science Supporting Stem Cell Research Invited Presentation Sanford Consortium for Regenerative.

The Missing Link: Dedicated End-to-End 10Gbps Optical Lightpaths for Clusters, Grids, and Clouds Invited Keynote Presentation 11 th IEEE/ACM International.

“End-to-end Optical Fiber Cyberinfrastructure for Data-Intensive Research: Implications for Your Campus” Featured Speaker EDUCAUSE 2010 Anaheim Convention.

"Cyberinfrastructure for Environmental Observations" Invited Talk to Symposium on Science and Technology in GEOSS: The Role of Universities Hosted by

Sequencing Genomics: The New Big Data Driver IntermezzoTalk SURFnet7, Part of GigaPort3 Utrecht, Netherlands December 7, 2011 Dr. Larry Smarr Director,

A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research Seminar Presentation Princeton Institute for Computational.

Calit2: Past, Present, and Future University Librarians Advisory Board Luncheon Seminar UC San Diego Library January 4, 2012 Dr. Larry Smarr Director,

High Performance Cyberinfrastructure Enabling Data-Driven Science in the Biomedical Sciences Joint Presentation UCSD School of Medicine Research Council.

Calit2's UCSD Building – A "Living Laboratory" For The Future

Creating High Performance Lambda Collaboratories" ONR Briefing ACCESS DC Arlington, VA March 25, 2005 Dr. Larry Smarr Director, California Institute for.

The Strongly Coupled LambdaCloud Tour TTI-Vanguard February 20, 2009 Dr. Larry Smarr Director, California Institute for Telecommunications.

High Performance Cyberinfrastructure Required for Data Intensive Scientific Research Invited Presentation National Science Foundation Advisory Committee.

Uses of the OptIPortal Presentation to the Minority Serving Institutions Cyberinfrastructure Empowerment Coalition June 10, 2010 Dr. Larry.

Calit2-Living in the Future " Keynote Sharecase 2006 University of California, San Diego March 29, 2006 Dr. Larry Smarr Director, California Institute.

Supercomputers and Supernetworks are Transforming Research Invited Talk Computing Research that Changed the World: Reflections and Perspectives Washington,

Bringing Mexico Into the Global LambdaGrid Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber.

Introduction to the UCSD Division of Calit2" Calit2 Tour NextMed / MMVR20 UC San Diego February 20, 2013 Dr. Larry Smarr Director, California.

Creating a Community Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis (a.k.a. CAMERA) Invited Talk Honoring David Kingsbury.

High Performance Cyberinfrastructure Enables Data-Driven Science in the Globally Networked World Keynote Presentation Sequencing Data Storage and Management.

Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis (CAMERA) Invited Talk CONNECT Board Meeting La Jolla, CA April 26, 2006.

Overview of Photonics Research at Calit2: Scaling from Nanometers to the Earth UCSD-NICT Joint Symposium on Innovative Lightwave, Millimeter-Wave and THz.

High Resolution Multimedia in a Ultra Bandwidth World After Dinner Talk IEEE ISM2005 Irvine, CA December 13, 2005 Dr. Larry Smarr Director, California.

Building a Global Collaboration System for Data-Intensive Discovery Distinguished Lecture Hawaii International Conference on System Sciences (HICSS-44)

Calit2: Experiments in Living in the Virtual/Physical World Invited Talk Cultivating Networked Centers of Excellence CineGrid International Workshop 2010.

Why Optical Networks Are Emerging as the 21 st Century Driver Scientific American, January 2001.

The OptIPuter and Its Applications Invited Talk IEEE/LEOS Summer 2009 Topicals Meeting on Future Global Networks July 27, 2009 Dr. Larry Smarr Director,

Scaling-Up the BIRN Building on the BIRN Workshop National Institutes of Health Bethesda, MD March 22, 2004 Dr. Larry Smarr Director, California Institute.

Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * Calit2 * LBNL * NICS * ORNL * SDSC Report to the Dept. of Energy Advanced Scientific.

1 Cyberinfrastructure Framework for 21st Century Science & Engineering (CIF21) NSF-wide Cyberinfrastructure Vision People, Sustainability, Innovation,

A High-Performance Campus-Scale Cyberinfrastructure: The Technical, Political, and Economic Presentation by Larry Smarr to the NSF Campus Bridging Workshop.

Computer Supported Cooperative Work: Past, Present, and Future Vision Panel on Computer Supported Cooperative Work 9 th Annual ON*VECTOR International.

Genomics at the Speed of Light: Understanding the Living Ocean The Gordon and Betty Moore Foundation 2nd Annual Marine Microbiology Investigator Symposium.

“High Performance Cyberinfrastructure Enables Data-Driven Science in the Globally Networked World” Invited Speaker Grand Challenges in Data-Intensive Discovery.

Science and Cyberinfrastructure in the Data-Dominated Era Symposium #1610, How Computational Science Is Tackling the Grand Challenges Facing Science and.

Building a Community Cyberinfrastructure to Support Marine Microbial Ecology Metagenomics Center for Earth Observations and Applications Advisory Committee.

CI Days: Planning Your Campus Cyberinfrastructure Strategy Russ Hobby, Internet2 Internet2 Member Meeting 9 October 2007.

Source: Jim Dolgonas, CENIC CENIC is Removing the Inter-Campus Barriers in California ~ $14M Invested in Upgrade Now Campuses Need to Upgrade.

“An Integrated Science Cyberinfrastructure for Data-Intensive Research” Panel CISCO Executive Symposium San Diego, CA June 9, 2015 Dr. Larry Smarr Director,

“Calit2: A UC Experiment for Living in the Future" Talk to UCSD Near You La Jolla, CA April 11, 2006 Dr. Larry Smarr Director, California Institute.

“Creating a High Performance Cyberinfrastructure to Support Analysis of Illumina Metagenomic Data” DNA Day Department of Computer Science and Engineering.

“Comparative Human Microbiome Analysis” Remote Video Talk to CICESE Big Data, Big Network Workshop Ensenada, Mexico October 10, 2013 Dr. Larry Smarr Director,

What is Cyberinfrastructure? Russ Hobby, Internet2 Clemson University CI Days 20 May 2008.

Cal-(IT) 2 : A Public-Private Partnership in Southern California U.S. Business Council for Sustainable Development Year-End Meeting December 11, 2003 Institute.

Chicago/National/International OptIPuter Infrastructure Tom DeFanti OptIPuter Co-PI Distinguished Professor of Computer Science Director, Electronic Visualization.

“Cyberinfrastructure for Ocean Cabled Observatories" Invited Talk NEPTUNE Regional Cabled Ocean Observatory Workshop Seattle, WA November 15, 2005 Dr.

Using Photonics to Prototype the Research Campus Infrastructure of the Future: The UCSD Quartzite Project Philip Papadopoulos Larry Smarr Joseph Ford Shaya.

SoCal Infrastructure OptIPuter Southern California Network Infrastructure Philip Papadopoulos OptIPuter Co-PI University of California, San Diego Program.

A High-Performance Campus-Scale Cyberinfrastructure For Effectively Bridging End-User Laboratories to Data-Intensive Sources Presentation by Larry Smarr.

Ocean Sciences Cyberinfrastructure Futures Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technologies Harry E.

The OptIPuter Project Tom DeFanti, Jason Leigh, Maxine Brown, Tom Moher, Oliver Yu, Bob Grossman, Luc Renambot Electronic Visualization Laboratory, Department.

The PRPv1 Architecture Model Panel Presentation Building the Pacific Research Platform Qualcomm Institute, Calit2 UC San Diego October 16, 2015.

“ Collaborations Between Calit2, SIO, and the Venter Institute—a Beginning " Talk to the UCSD Representative Assembly La Jolla, CA November 29, 2005 Dr.

“CAMERA Goes Live!" Presentation with Craig Venter National Press Club Washington, DC March 13, 2007 Dr. Larry Smarr Director, California Institute for.

Cyberinfrastructure Overview Russ Hobby, Internet2 ECSU CI Days 4 January 2008.

“The UCSD Big Data Freeway System” Invited Short Talk Workshop on “Enriching Human Life and Society” UC San Diego February 6, 2014 Dr. Larry Smarr Director,

SIU Information Technology April 28, 2015 Research Computing and Cyberinfrastructure.

High Performance Cyberinfrastructure Discovery Tools for Data Intensive Research Larry Smarr Prof. Computer Science and Engineering Director, Calit2 (UC.

“OptIPuter: From the End User Lab to Global Digital Assets" Panel UC Research Cyberinfrastructure Meeting October 10, 2005 Dr. Larry Smarr.

Lennart Johnsson Professor CSC Director, PDC

Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * Calit2 * LBNL * NICS * ORNL * SDSC Report to the Dept. of Energy Advanced.

The OptIPortal, a Scalable Visualization, Storage, and Computing Termination Device for High Bandwidth Campus Bridging Presentation by Larry Smarr to.

Presentation transcript:

High Performance Cyberinfrastructure Discovery Tools for Data Intensive Research Larry Smarr Prof. Computer Science and Engineering Director, Calit2 (UC San Diego/UC Irvine)

AbstractAbstract High performance cyberinfrastructure (10Gbps dedicated optical channels end-to-end) enable new levels of discovery for data-intensive research projectssuch as cosmological simulations, ocean observing, and microbial metagenomics. In addition to the national optical fiber infrastructure provided by National LamdbaRail, we need local campus high performance research cyberinfrastructure to provide on- ramps, as well as compute and storage clouds, to augment the emerging remote commercial clouds.

Dedicated 10,000Mbps (10Gbps) Supernetworks Enable Remote Visual Analysis of Big Data NLR 80 x 10Gb Wavelengths National LambdaRail Interconnects Two Dozen State and Regional Optical Networks Also Dynamic Circuit Network Is Now Available

NSFs OptIPuter Project: Using Supernetworks to Meet the Needs of Data-Intensive Researchers OptIPortal– Termination Device for the OptIPuter 10Gbps Backplane

Exploring Cosmology With Supercomputers, Supernetworks, and Supervisualization Supercomputer Output –148 TB Movie Output (0.25 TB/file) –80 TB Diagnostic Dumps (8 TB/file) Connected at 10Gbps –Oak Ridge to ANL to SDSC Science: Norman, Harkness, Paschos, SDSC Visualization: Insley, ANL; Wagner SDSC ANL * Calit2 * LBNL * NICS * ORNL * SDSC Intergalactic Medium on 2 Billion Light Year Scale

End-to-End 10Gbps Cyberinfrastructure for Petascale High Performance Computing End Users Mike Norman, SDSC Using OptIPortal to Analyze Petascale Cosmological Big Data log of gas temperature log of gas density

Calit2 Microbial Metagenomics Cluster-Users Can Connect by Shared Internet or 10Gbps Optical Paths 512 Processors ~5 Teraflops ~ 200 Terabytes Storage Source: Phil Papadopoulos, SDSC, Calit2 Nearly 4000 Users Over 75 Countries

Using 10 Gbps Big Data Access and Analysis- Collaboration Between Calit2 and U Washington Ginger Armbrusts Diatom Chromosomes Photo Credit: Alan Decker Feb. 29, 2008 iHDTV: 1500 Mbits/sec Calit2 to UW Over NLR UW Research Channel

MIT Using OptIPortal to Analyze 10km Coupled Ocean Microbial Simulation MITs Ed DeLong & Darwin Project Team

The NSF-Funded Ocean Observatory Initiative– a Complex System of Systems Cyberinfrastructure Source: Matthew Arrott, Calit2 Program Manager for OOI CI

Taking Sensornets to the Ocean Floor: Remote Interactive HD Imaging of Deep Sea Vent 1 cm. Source: John Delaney and Research Channel, University of Washington

NSF OOI is a $400M Program -OOI CI is $34M Part of OOI Source: Matthew Arrott, Calit2 Program Manager for OOI Cyberinfrastructure 30 Software Engineers Housed at Science Program 25 to 30 Years Construction Program 5 Years

OOI CI is Built on National LambdaRails and Internet2s DCN Optical Infrastructure Source: John Orcutt, Matthew Arrott, SIO/Calit2

High Definition Video Connected OptIPortals: Virtual Working Spaces for Data Intensive Research Source: Falko Kuester, Kai Doerr Calit2; Michael Sims, NASA

Analyzing Big Data in 3D Stereo: The NexCAVE OptIPortal Source: Tom DeFanti, Array of JVC HDTV 3D LCD Screens Calit2s KAUST NexCAVE = 22.5MPixels

Blueprint for the Digital University--Report of the UCSD Research Cyberinfrastructure Design Team research.ucsd.edu/documents/rcidt/RCIDTReportFinal2009.pdf April 24, 2009 DataOasis (Central) Storage OptIPortal Campus Lab Cluster Digital Data Collections Triton – Petadata Analysis Gordon – HPC System Cluster Condo Scientific Instruments N x 10Gbe CENIC, NLR, I2DCN Source: Philip Papadopoulos, SDSC, Calit2 UCSD

California and Washington Universities Are Testing a 10Gbps Connected Commercial Data Cloud Amazon Experiment for Big Data –Only Available Through CENIC and Pacific NW GigaPOP Private 10Gbps Peering Paths –Includes Amazon EC2 Computing and S3 Storage Services Amazon Experiment for Big Data –Only Available Through CENIC and Pacific NW GigaPOP Private 10Gbps Peering Paths –Includes Amazon EC2 Computing and S3 Storage Services

You Can Download This Presentation at lsmarr.calit2.net