The OptIPuter and NLR Tom DeFanti, Maxine Brown, Jason Leigh, Alan Verlo, Linda Winkler, Joe Mambretti Chicago Larry Smarr, Mark Ellisman, Phil Papadopoulos, Greg Hidley San Diego Ron Johnson, Dave Richardson Seattle

What is the OptIPuter? Optical networking, Internet Protocol, computer storage, processing and visualization technologies Tightly couples computational resources over parallel optical networks using the IP communication mechanism The OptIPuter exploits a new world in which the central architectural element is optical networking, not computers - creating "supernetworks"

What is the OptIPuter? The goal of this new architecture is to enable scientists who are generating terabytes and petabytes of data to interactively visualize, analyze, and correlate their data from multiple storage sites connected to optical networks.

Why Now? e-Science requires OptIPuter capabilities –Many 2D GigaPixel and 3D GigaZone data objects –Cyberinfrastructure plans are being made-influence designs NOW! We can build on Global Grid Research –Grid Middleware has much of what we need –But Grid is built on stochastic foundation “Best Effort” Internet means unpredictable latency –Need determinism: predictable reservable lambdas Availability of Technology for Dedicated Light Pipes –State, National, and International Dark Fiber Nets Being Turned-On –Lots of Parallel High Bandwidth Clusters for Endpoints of Users –Shakeout of Optical Switch Market 30  2 Companies, Both OptIPuter Anchors, Inexpensive (~$1000/port) –Starlight/PacificWave Have Driven Global Lambda Connectivity –Cost of Bandwidth not the bottleneck anymore

George Seweryniak U.S. Department of Energy “Optical Networks are critical to the Federal Agencies –Need to closely work together with private industry on development to deployment –Federal agencies have stepped up to the plate Current Science trends need them now No sign of abating requirements for big pipes and dynamic provisioning”

10GE CAVEwave on the National LambdaRail Map Source: John Silvester, Dave Reese Tom West, Ron Johnson I-WIRE in Illinois Connects EVL and NCSA to StarLight and CAVEwave Connects StarLight in Chicago to Seattle and then Joins Pacific WAVE to Cal-(IT) 2 in San Diego and Irvine (later Bay Area and Los Angeles) for OptIPuter/GLIF Experiments. EVL

What’s a 10GE CAVEwave Cost? Less than a 1GE Internet connection About what a network engineer or full professor costs, fully loaded Much less than the computers at each end to keep it busy Much less than the router capacity needed at each end to accept the 10GE Your institution needs to be a NLR member (priceless)

European lambdas to US (red) –10Gb Amsterdam—Chicago –10Gb London—Chicago –10Gb Amsterdam—NYC Canadian lambdas to US (white) –30Gb Chicago-Canada-NYC –30Gb Chicago-Canada-Seattle US sublambdas to Europe (grey) –6Gb Chicago—Amsterdam Japan JGN II lambda to US (cyan) –10Gb Chicago—Tokyo European lambdas (yellow) –10Gb Amsterdam—CERN –2.5Gb Prague—Amsterdam –2.5Gb Stockholm—Amsterdam –10Gb London—Amsterdam IEEAF lambdas (blue) –10Gb NYC—Amsterdam –10Gb Seattle—Tokyo CAVEwave/PacificWave (purple) –10Gb Chicago—Seattle –10Gb Seattle—LA—San Diego –10Gb Seattle—LA Northern Light UKLight CERN Japan Actual TransLight Experimental Lit up Lambdas Today PNWGP Manhattan Landing

OptIPuter Experiment #1 Wide-Area Vol-a-Tile and JuxtaView Applications –Electronic Visualizaton Laboratory (EVL), University of Illnois at Chicago –University of Amsterdam JuxtaView displays ultra-high resolution 2D images, such as USGS maps. It invokes LambdaRAM, which pre-fetches portions of large datasets before an application needs them and stores the data in the local cluster's memory for display. Vol-a-Tile dynamically retrieves, renders and displays large volumetric datasets from remote storage. It invokes OptiStore, which extracts relevant information from raw volumetric datasets and produces visual objects for display. Aggressive pre-fetching and large bandwidth utilization can overcome network latency that hinders interactive applications. Datasets are stored on a cluster at University of Amsterdam, streamed on demand over a 4Gbps transatlantic link, and displayed on EVL’s GeoWall2 visualization cluster.

30 Megapixel Viewport into a 10 GigaPixel Dataset

OptIPuter Experiment #2 Terabyte Data Juggling and DVC Framework over the OptIPuter Network –Concurrent Systems Architecture Group (CSAG), UCSD –Cal-(IT) 2 /JSOE, UCSD Group Transport Protocol (GTP) is a transport protocol that efficiently manages receiver contention likely to arise in high-bandwidth networks. GTP achieves both high-transfer bandwidth for single flows and also efficient sharing and coordination among multiple convergent flows, enabling efficient data fetching from multiple distributed data sources. A multi-gigabyte SIO dataset is moved across four UCSD OptIPuter node sites and between UIC and Amsterdam. Dynamic Virtual Computer (DVC) middleware ties together novel configurable optical network capabilities with traditional Grid resources. Using an interactive GUI, a prototype implementation of DVC shows dynamic resource aggregation – resource discovery, selection and binding – as well as the running of a message-passing application across these resources. Resources are assembled from several UCSD sites and from University of Amsterdam. www-csag.ucsd.edu/projects/Optiputer.html

OptIPuter Experiment #3 Grid-Based Visualization Pipeline for OptIPuter Clusters –Information Sciences Institute (ISI), University of Southern California –Electronic Visualization Laboratory (EVL), UIC The Grid Visualization Utility (GVU) facilitates the construction of scalable visualization pipelines, based on the Globus Toolkit. GVU is used to enable real-time interactive viewing of high-resolution time-series volume datasets. Current efforts examine how load-balancing issues can be alleviated by fine-grained decomposition and distribution of datasets across clusters and other distributed compute resources. GVU currently supports interactive viewing of 3D structures culled from large datasets on a single display; Bolivia Earthquake data is used. Future work will focus on tiled displays, such as GeoWall2, for distributing the rendering load to handle the interactive viewing of multiple, complex structures.

OptIPuter Experiment #4 Trans-Pacific HDTV Feedback & Remote-Control Scenarios of Remote Instrumentation –National Center for Microscopy and Imaging Research (NCMIR) and Biomedical Informatics Research Network (BIRN), UCSD –Osaka University –KDDI R&D Labs NCMIR researchers demonstrate live streaming HDTV from the world’s largest microscope in Osaka, Japan. The video data is streamed to UIC/EVL and UCSD while being controlled by project scientists in San Diego. High-quality video is essential for resolving useful information, such as changes in gradients in a high- noise, low-contrast environment. HDTV combined with dedicated lambdas will provide lower latencies and control of network jitter, especially important in these large streams of video data. This step is the first in a data acquisition feedback loop for instrumentation steering, control, computation and visualization.

OptIPuter Experiment #5 Application-Controlled Light-Path Provisioning over Multi-Domain OptIPuter Environments –Electronic Visualizaton Laboratory (EVL), University of Illnois at Chicago –Int’l. Center for Advanced Internet Research (iCAIR), Northwestern Univ. –University of Amsterdam (UvA) TeraVision streamed fractal animation by Dan Sandin, EVL In this demonstration, end-to-end light paths are provisioned across multiple domains, and then local domain controllers are invoked to access and set up optical switches, using the following tools: EVL’s Photonic Inter-domain Negotiator (PIN) EVL’s Photonic Data Controller (PDC) iCAIR’s Optical Dynamic Intelligent Network (ODIN) over Chicago optical metro OMNInet UvA’s Inter-domain Generic Authorization, Authentication, and Accounting (AAA) procedures Using EVL’s TeraVision application for capturing and streaming high-resolution computer graphics over gigabit networks, an animation is streamed, first between two domains (UvA to EVL) over multi-gigabit transoceanic links, and then among three domains (UvA, NU via OMNInet, and EVL).

Multi-Domain Lightpaths Photonic Interdomain Negotiator OC-192 Signalling Link StarLight OMNInet All-optical MAN All-optical LAN PDC All-optical LAN Cluster All-optical LAN PIN Cluster BOD/AAA Cluster (Chicago) PIN ODIN/GMPLS University of Illinois at Chicago University of Amsterdam NetherLight (Amsterdam) PDC Chicago and Northwestern at Evanston

OptIPuter Experiment #6 JuxtaView, Vol-a-Tile and GVU at SIO and SIO Visual Objects Distribution –Scripps Institution of Oceanography (SIO), UCSD –SDSC, UCSD –Cal-(IT) 2 /JSOE, UCSD –Electronic Visualizaton Laboratory (EVL), University of Illnois at Chicago –Information Sciences Institute (ISI), University of Southern California This demonstration showcases OptIPuter applications that display high-resolution imagery (JuxtaView), volume rendering (Vol-a-Tile and GVU) and scene files composed of heterogeneous geologic datasets (SIO Visual Objects). Here, SIO scientist Debi Kilb interacts with IKONOS satellite imagery using JuxtaView (on left panel) and a scene file that combines the same imagery with topography, bathymetry and seismic images (displayed on right panel). Both datasets are fetched over the campus OptIPuter fiber from the UCSD storage cluster.

OptIPuter Experiment #7 Online Brain Maps: Deposition, Distribution and Visualization of Large-Scale Brain Maps in a Near-Real-Time Environment –National Center for Microscopy and Imaging Research (NCMIR) and Biomedical Informatics Research Network (BIRN), UCSD –SDSC, UCSD –Cal-(IT) 2 /JSOE, UCSD NCMIR researchers demonstrate a system within the Telescience Portal that allows users to do biological studies involving electron microscopic tomography by guiding them through the process, from acquisition to analysis. GridFTP is used to distribute very large (>1Gb) brain maps from the UCSD multi-photon microscope to other OptIPuter on-line sites in Southern California (UCSD, UCI, SDSU, and USC). Data is sent from the OptIPuter’s IBM storage cluster to local resources, such as the Geowall2 or computational resources. The goal is to be able to transfer data from an instrument to OptIPuter-connected resources in real time, enabling researchers to steer the data acquisition process and monitor progress, which can currently take as long as 22 days.

The Crisis Response Room of the Future SHD Streaming TV -- Immersive Virtual Reality – 100 Megapixel Displays

The Living Room of 2010? The CineGrid Home Office and Game Room

Global Lambda Integrated Facility GLIF World Map – December 2004 Predicted international Research & Education Network bandwidth, to be made available for scheduled application and middleware research experiments by December Visualization by Bob Patterson, NCSA.

Global Lambda Integrated Facility Bandwidth for Experiments, December Visualization by Bob Patterson, NCSA.

Global Lambda Integrated Facility Bandwidth for Experiments, December Visualization by Bob Patterson, NCSA.

September 26-30, 2005 University of California, San Diego California Institute for Telecommunications and Information Technology [Cal-(IT) 2 ] United States Announcing… i Grid 2 oo 5 T H E G L O B A L L A M B D A I N T E G R A T E D F A C I L I T Y

Thank You! TransLight planning, research, collaborations, and outreach efforts are made possible, in major part, by funding from: –National Science Foundation (NSF) awards SCI , SCI , CNS , CNS , SCI , and CNS –State of Illinois I-WIRE Program, and major UIC cost sharing –Northwestern University for providing space, power, fiber, engineering and management –Pacific Wave, StarLight, National LambdaRail, CENIC, PNWGP, CANARIE, SURFnet, UKERNA, and IEEAF for Lightpaths DoE/Argonne National Laboratory for StarLight and I-WIRE network engineering and design