Pacific Rim Application and Grid Middleware Assembly (PRAGMA)1: Philip Papadopoulos, Ph.D. Chief Technology Officer, San Diego Supercomputer Center Research Scientist, Calit2 University of California San Diego Structural characteristics of community of practices are redefined to a domain of knowledge, a notion of community, and a practice  Community of Practice - Domain of knowledge – creates common ground, inspires members to participate, guides their learning and gives meaning to their action - Community – Notion of community creates social fabric for learning. Strong community fosters interactions / encourages willingness to share ideas - Practice - while the domain provides the general area of interest for the community, the practice is the specific focus around which the community develops, shares and maintains its core of knowledge The Pacific Rim Applications and Grid Middleware Assembly (PRAG­MA), established in 2002, is a robust, international network of research scientists from more than 30 institutions who address shared science and cyberin­frastructure challenges   PRAGMA pursues activities in four broad interdependent areas Fostering international scientific expeditions by forging teams of domain scientists and cyberinfrastructure researchers who develop and test necessary information technologies to solve specific scientific questions and create usable, international-scale, cyber environments; Developing and improving a grassroots, international cyberinfrastructure for testing, computer science insight and advancing scientific applications by sharing resources, expertise, and software; Infusing new ideas by developing new researchers with experience in cross-border science and by continuing to engage strategic partners; Building and enhancing the essential people-to-people trust and organization developed through regular face-to-face meetings—a core component of PRAGMA’s success. This talk will provide an overview of PRAGMA and highlight its accomplishments in its twelve years since inception, and will focus two current the scientific “virtual” expeditions, one on predicting freshwater quality in lake, the other on biodiversity in extreme environments. The talk will give some lesson’s learned in creating a distributed yet coordinated network of researchers, give examples from PRAGMA and from another ecological network, the Global Lakes Ecological Observatory Network, and mention opportunity for active student engagement in these networks. One goal for the talk is to explore how PRAGMA might help stimulate a discussion around such a network in biodiversity work in Southeast Asia and beyond. 1 US Participation funded by NSF Award OCI-1234983

PRAGMA Members and Affiliates Community of Practice Scientific Expeditions and Infrastructure Experiments for Pacific Rim Institutions and Researchers PRAGMA Members and Affiliates Why Did we start PRAGMA? - To realize a vision of technology working together - Means people have to work together! For individuals new to a PRAGMA workshop – welcome! As science becomes more distributed and collaborative, there are small to medium sized groups, distributed, that want to work together. PRAGMA is about enabling this long tail of science, through scientific expeditions and infrastructure experiments. We have always focused on the Pacific Rim. You can see the institutions involved. This includes the four new sites that were accepted as members at PRAGMA 24 Established in 2002

PRAGMA’s Expedition Model of Collaboration Domain Scientists Long-Term sustained effort to answer scientific question(s) through co-design Technology developers Lake Ecology: understand the processes that govern lake eutrophication Challenge: model lakes (simulation) with large ranges of inputs to understand algal blooms and fresh-water quality Biodiversity: understand how patterns of specifies diversity emerge and how they are sustained Challenge: Make Lifemapper simulation cluster portable to allow greater use by collaborators with restricted data PRAGMA Experimental Network Testbed (ENT): explore of software defined networking in international context Challenge: understand and evaluate Software Defined Networking capabilitieis at international scale Expeditions PA Thoughts about collaboration How do I know what to build if I’m not talking deeply with potential users? How can I understand their pain points if I’m not willing to site down and listen?

It’s NOT the capacity of the network – It’s connectivity! In general, data in the PRAGMA Context isn’t “big” Biodiversity data sets are: O(10s of GBs) By Policy, may need to be kept “in place” Difficult to integrate into a common spatial frames Species observation data Climate simulation output data Land use data Satellite observations Lake ecology data sets are: Real world sensor inputs In-person observations (e.g. Lake Observer App on Play Store/App Store) Simulation output from 1D model  Data can be shared, but limited sharing is normal

Virtual images sharing Building on the International Development of PRAGMA’s Multi-Cloud Testbed Goal: a persistent cloud testbed for PRAGMA members to run experiments. Raw image Universal Clonezilla Live VM Pragma_boot Local Virtual Images Repository Tools: CZISO - enables sharing virtual cluster images, leverages UCSD’s unlimited Google storage Clonezilla (NCHC, Taiwan) to convert images to different formats (raw, qcow, zvol) Cloud Scheduler - enhanced interface to boot/manage virtual clusters Cloud Scheduler NCHC Taiwan AIST Japan NAIST UCSD USA Virtual images sharing Needed - Mention of how GRAPLEr Relates - Mention of how this drives / uses technologies developed elsewhere in PRAGMA - Mention of international sites / partners involved Pragma_boot – deploys virtual cluster images across PRAGMA physical sites using local network configuration settings

Our Approach Use “Overlay” Networks to provide a trusted environment for focused sharing of resources

General Lake Model (GLM ) through GRAPLEr GRAPLEr Web Service Expose simple R desktop interface to users Handle packaging of multiple simulations into Condor job, compression, job submission, post-processing receive Web service HTCondor send Internet R, plus GRAPLEr library IPOP overlay network UF VT … GRAPLEr UW Azure PRAGMA GRAPLEr: A Lake Ecology Distributed Collaborative Environment Integrating Overlay Networks, High-throughput Computing and Web Services, Subratie, K, Aditya, S, Figueiredo, RJ, Carey, C, Hanson, P

Impacts Branch of fresh-water ecology dramatically changed From statistical analysis only to analysis + simulation 10K+ simulations (parameter sweep) for a 5 year analysis (accomplished overnight with HTCondor) Capability being brought into the classroom by PI Cayelan Carey at Virginia Tech Also experimenting in a hybrid networking space

Overlay and SDN Hybrid for Seamless High Performance Virtual Network Kyuho Jeong (UF) Kohei Ichikawa (NAIST) Renato Figueiredo (UF)

IPOP Recap (nested) VM1 eth0 (nested) VM2 eth0 IPOP (nested) VM3 eth0 (nested) VM4 eth0 Create direct P2P virtual tunnel across multiple NAT/Firewall using ICE protocol Each virtual tunnel is mapped to either MAC or IP address Tap device to interface with O/S network stack Packets are encapsulated by IP/UDP header. Runs in application layer N2N Encryption (DTLS – Datagram Transport Layer Sec) IPOP IPOP TAP IPOP TAP veth0 veth1 veth0 veth1 L2 Bridge L2 Bridge NAT eth0 NAT eth0 Host Host NAT NAT IPOP Overlay

IPOP Deployment across Internet VM container container container VM container container container Layer 2 Virtualization Network Virtual layer Physical layer Amazon EC2 container IPOP container IPOP Open vSwitch Compute instance (VM) Open vSwitch Compute instance (VM) container IPOP Open vSwitch Cloud Network (Layer 3) Compute instance (VM) Cloud Network (Layer3) IPOP Public Internet container Open vSwitch Google Compute Engine Compute instance (VM) Overlay Datapath Compute instance (VM) Compute instance (VM) SDN Datapath Cloud Network (Layer 3) PRAGMA Cloud/Cloudlab

PRAGMA on the Web Home: http://www.pragma-grid.org Github: http://github.com/pragmagrid