TeraGrid Science Gateways Nancy Wilkins-Diehr TeraGrid Area Director for Science Gateways TeraGrid Rount Table, October 7, 2010.

Slides:



Advertisements
Similar presentations
1 US activities and strategy :NSF Ron Perrott. 2 TeraGrid An instrument that delivers high-end IT resources/services –a computational facility – over.
Advertisements

Gateway Transition Issues TeraGrid 10, August 2-5, 2010.
User Introduction to the TeraGrid 2007 SDSC NCAR TACC UC/ANL NCSA ORNL PU IU PSC.
RCAC Research Computing Presents: DiaGird Overview Tuesday, September 24, 2013.
CyberGIS and TeraGrid Science Gateways update Nancy Wilkins-Diehr TeraGrid Area Director for Science Gateways TeraGrid Quarterly, December.
Core Services I & II David Hart Area Director, UFP/CS TeraGrid Quarterly Meeting December 2008.
Network, Operations and Security Area Tony Rimovsky NOS Area Director
Project Proposal: Academic Job Market and Application Tracker Website Project designed by: Cengiz Gunay Client: Cengiz Gunay Audience: PhD candidates and.
Attribute-based Authentication for Gateways Jim Basney Terry Fleury Stuart Martin JP Navarro Tom Scavo Jon Siwek Von Welch Nancy Wilkins-Diehr.
TeraGrid Gateway User Concept – Supporting Users V. E. Lynch, M. L. Chen, J. W. Cobb, J. A. Kohl, S. D. Miller, S. S. Vazhkudai Oak Ridge National Laboratory.
Creating the CIPRES Science Gateway for Inference of Large Phylogenetic Trees Mark A. Miller San Diego Supercomputer Center.
GIG Software Integration: Area Overview TeraGrid Annual Project Review April, 2008.
Ian Fisk and Maria Girone Improvements in the CMS Computing System from Run2 CHEP 2015 Ian Fisk and Maria Girone For CMS Collaboration.
Data Management Subsystem: Data Processing, Calibration and Archive Systems for JWST with implications for HST Gretchen Greene & Perry Greenfield.
National Center for Supercomputing Applications The Computational Chemistry Grid: Production Cyberinfrastructure for Computational Chemistry PI: John Connolly.
Cloud Usage Overview The IBM SmartCloud Enterprise infrastructure provides an API and a GUI to the users. This is being used by the CloudBroker Platform.
TeraGrid Science Gateways: Scaling TeraGrid Access Aaron Shelmire¹, Jim Basney², Jim Marsteller¹, Von Welch²,
GLAST LAT ProjectDOE/NASA Baseline-Preliminary Design Review, January 8, 2002 K.Young 1 LAT Data Processing Facility Automatically process Level 0 data.
Software for Science Gateways: Open Grid Computing Environments Marlon Pierce, Suresh Marru Pervasive Technology Institute Indiana University
SAN DIEGO SUPERCOMPUTER CENTER NUCRI Advisory Board Meeting November 9, 2006 Science Gateways on the TeraGrid Nancy Wilkins-Diehr TeraGrid Area Director.
Future role of DMR in Cyber Infrastructure D. Ceperley NCSA, University of Illinois Urbana-Champaign N.B. All views expressed are my own.
Stephen Booth EPCC Stephen Booth GridSafe Overview.
National Center for Supercomputing Applications University of Illinois at Urbana-Champaign The Dark Energy Survey Middleware LSST Workflow Workshop 09/2010.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Collaborating with iPlant.
UltraScan Gateway Advanced Support GIG Team: Suresh Marru, Raminder Singh, Marlon Pierce Pervasive Technology Institute Indiana University Gateway Personal:
Wenjing Wu Computer Center, Institute of High Energy Physics Chinese Academy of Sciences, Beijing BOINC workshop 2013.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Collaborating with iPlant.
CSIU Submission of BLAST jobs via the Galaxy Interface Rob Quick Open Science Grid – Operations Area Coordinator Indiana University.
Accelerating Scientific Exploration Using Workflow Automation Systems Terence Critchlow (LLNL) Ilkay Altintas (SDSC) Scott Klasky(ORNL) Mladen Vouk (NCSU)
1 PY4 Project Report Summary of incomplete PY4 IPP items.
The Future of the iPlant Cyberinfrastructure: Coming Attractions.
Instrumentation of the SAM-Grid Gabriele Garzoglio CSC 426 Research Proposal.
TeraGrid Quarterly Meeting Dec 5 - 7, 2006 Data, Visualization and Scheduling (DVS) Update Kelly Gaither, DVS Area Director.
Turning science problems into HTC jobs Wednesday, July 29, 2011 Zach Miller Condor Team University of Wisconsin-Madison.
TeraGrid CTSS Plans and Status Dane Skow for Lee Liming and JP Navarro OSG Consortium Meeting 22 August, 2006.
Large Scale Nuclear Physics Calculations in a Workflow Environment and Data Provenance Capturing Fang Liu and Masha Sosonkina Scalable Computing Lab, USDOE.
 Apache Airavata Architecture Overview Shameera Rathnayaka Graduate Assistant Science Gateways Group Indiana University 07/27/2015.
Using SWARM service to run a Grid based EST Sequence Assembly Karthik Narayan Primary Advisor : Dr. Geoffrey Fox 1.
TeraGrid Advanced Scheduling Tools Warren Smith Texas Advanced Computing Center wsmith at tacc.utexas.edu.
The iPlant Collaborative Community Cyberinfrastructure for Life Science Tools and Services Workshop Discovery Environment Overview.
Scalable Systems Software for Terascale Computer Centers Coordinator: Al Geist Participating Organizations ORNL ANL LBNL.
Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.
Biomedical and Bioscience Gateway to National Cyberinfrastructure John McGee Renaissance Computing Institute
TeraGrid Extension Gateway Activities Nancy Wilkins-Diehr TeraGrid Quarterly, September 24-25, 2009 The Extension Proposal!
1 NSF/TeraGrid Science Advisory Board Meeting July 19-20, San Diego, CA Brief TeraGrid Overview and Expectations of Science Advisory Board John Towns TeraGrid.
TeraGrid Gateway User Concept – Supporting Users V. E. Lynch, M. L. Chen, J. W. Cobb, J. A. Kohl, S. D. Miller, S. S. Vazhkudai Oak Ridge National Laboratory.
Biomedical and Bioscience Gateway to National Cyberinfrastructure John McGee Renaissance Computing Institute
NOS Report Jeff Koerner Feb 10 TG Roundtable. Security-wg In Q a total of 11 user accounts and one login node were compromised. The Security team.
ApproxHadoop Bringing Approximations to MapReduce Frameworks
Data, Visualization and Scheduling (DVS) TeraGrid Annual Meeting, April 2008 Kelly Gaither, GIG Area Director DVS.
Network, Operations and Security Area Tony Rimovsky NOS Area Director
SPI NIGHTLIES Alex Hodgkins. SPI nightlies  Build and test various software projects each night  Provide a nightlies summary page that displays all.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views.
Attribute-based Authentication for Gateways Jim Basney Terry Fleury Stuart Martin JP Navarro Tom Scavo Nancy Wilkins-Diehr.
Software Integration Highlights CY2008 Lee Liming, JP Navarro GIG Area Directors for Software Integration University of Chicago, Argonne National Laboratory.
Northwest Indiana Computational Grid Preston Smith Rosen Center for Advanced Computing Purdue University - West Lafayette West Lafayette Calumet.
TG ’08, June 9-13, State of TeraGrid John Towns Co-Chair, TeraGrid Forum Director, Persistent Infrastructure National Center for Supercomputing.
Building PetaScale Applications and Tools on the TeraGrid Workshop December 11-12, 2007 Scott Lathrop and Sergiu Sanielevici.
Data Infrastructure in the TeraGrid Chris Jordan Campus Champions Presentation May 6, 2009.
Building on virtualization capabilities for ExTENCI Carol Song and Preston Smith Rosen Center for Advanced Computing Purdue University ExTENCI Kickoff.
TeraGrid’s Process for Meeting User Needs. Jay Boisseau, Texas Advanced Computing Center Dennis Gannon, Indiana University Ralph Roskies, University of.
Quarterly Meeting Spring 2007 NSTG: Some Notes of Interest Adapting Neutron Science community codes for TeraGrid use and deployment. (Lynch, Chen) –Geared.
INTRODUCTION TO XSEDE. INTRODUCTION  Extreme Science and Engineering Discovery Environment (XSEDE)  “most advanced, powerful, and robust collection.
TeraGrid Software Integration: Area Overview (detailed in 2007 Annual Report Section 3) Lee Liming, JP Navarro TeraGrid Annual Project Review April, 2008.
Scientific Data Processing Portal and Heterogeneous Computing Resources at NRC “Kurchatov Institute” V. Aulov, D. Drizhuk, A. Klimentov, R. Mashinistov,
Application Sharing Bhavesh Amin Casey Miller Casey Miller Ajay Patel Ajay Patel Bhavesh Thakker Bhavesh Thakker.
VisIt Project Overview
Tools and Services Workshop
Joslynn Lee – Data Science Educator
Storing and Accessing G-OnRamp’s Assembly Hubs outside of Galaxy
Presentation transcript:

TeraGrid Science Gateways Nancy Wilkins-Diehr TeraGrid Area Director for Science Gateways TeraGrid Rount Table, October 7, 2010

What have the gateways been up to? Ultrascan –Borries Demeler, UT ; Suresh Marru, Raminder Singh, IU Gateway software listing Wrap up of support for Arroyo, RENCI science portal –But hopefully not the end of TG usage by those groups Dark Energy Survey –Jim Myers, Michelle Gower, NCSA CUE presentation –Derek Simmel, PSC login-env, build, comm, math, tg TeraGrid Rount Table, October 7, 2010

GRAM5 –Making use of TG portal user forum for discussion –Interest in sharing experiences with OSG –Update on Inca tests (able to recreate load from “Gateway Debug 2007”) –Gateway experiences – hung processes when errors pile up –SGE job manager issues –Nice work by David Carver (TACC), Suresh Marru (IU), Stu Martin (ANL) Expressed Sequence Tag gateway –Archit Kulshrestha, IU CIPRES –Over 600 users on TG Apr-June –2.7M hours awarded 7/1/10, “model gateway proposal” But able to use much more than this Gateways in the extension year Gateway study TeraGrid Rount Table, October 7, 2010

Analytical Ultracentrifugation Emerging computational tool for the study of proteins Samples from researchers all over the world –Some (Germany, Australia) have their own ultracentrifuges and use only the analysis capabilities, others send samples to UT to spin Spin the samples at high speeds, learn about macromolecule properties Monte Carlo simulations Observations are electronically digitized and stored for further mathematical analysis TeraGrid Rount Table, October 7, 2010 Source: Suresh Marru, IU The Center for Analytical Ultracentrifugation of Macromolecular Assemblies, UT Health Sciences

Comprehensive data analysis environment Management of analytical ultracentrifugation data for single users or entire facilities Support for storage, editing, sharing and analysis of data –HPC facilities used for 2-D spectrum analysis and genetic algorithm analysis TeraGrid (~2M CPU hours used) Technische University of Munich Juelich Supercomputing Center Portable graphical user interface MySQL database backend for data management Over 30 active institutions TeraGrid Rount Table, October 7, 2010 Source: Suresh Marru, IU

Gateway and ASTA support a growing trend TeraGrid advanced support –Fault tolerance –Workflows –Use of multiple TG resources (using Lonestar, expanding to QueenBee and Ranger, using Quarry for test server, waiting for GRAM5 on Ranger) –Community account implementation –Remote steering –Improved UI (no manual specification of CPU time) –Applying lessons learned from GridChem, LEAD, incorporating new features into OGCE LEAD is portlet-based, Gridchem is java swing client side app, Ultrascan is php and perl-based gateway, all can use OGCE Big MPI app that forks off many independent runs, improvements here will be tackled by TG's advanced support team TeraGrid Rount Table, October 7, 2010 Source: Suresh Marru, IU

Gateway software listing Populate TeraGrid’s information service with gateway software information –Similar to RP software listings But, RP listings are maintained at RPs, IIS pulls from those sources With gateways we are thinking they fill in a form and push the info to IIS 887/gawsr-howto/ TeraGrid Rount Table, October 7, 2010

Dark Energy Survey Know universe is expanding, but expansion is accelerating for unknown reasons DES is telescope experiment to constrain various theories- 4m telescope in Chile, Fermi and others developing new lens, working with simulated data until telescope goes online in TB raw data over 5 years, 4 PB of derived products- lots of filtering Thousands of jobs run on TeraGrid each week with very few failures Removing light from bright stars, airplanes, clouds, calibration- telescope operated by staff, users will use the portal to do queries for particular stars/regions of the sky afterward TeraGrid Rount Table, October 7, 2010 Source: Jim Myers and Michele Gower, NCSA

Condor dagman, condor-g, pre-ws gram, gridftp, elf/ogrescript for monitoring (developed at ncsa), oracle Challenges –Efficiently managing small jobs in big batch world Databases stresses, block updates instead of individual transactions for better performance, indexing strategies, narrow vs wide tables ~100 front end users, expected to grow in production- changing paradigms from Sloan Digital Sky Survey - data now too large for bulk downloads and full table scans TeraGrid Rount Table, October 7, 2010 Source: Jim Myers and Michele Gower, NCSA

Expressed Sequence Tag (EST) Pipeline Integrate existing computational biology software Expand compute capacity by using TeraGrid Take raw genome data in the FASTA format and run a series of applications on it –RepeatMasker, PaCE, CAP3 and BLAST used to generate the final assembled output EST Pipeline based on the SWARM Web Service that provides a web service interface to clients and also manages the bulk job submission using the Birdbath API to submit to Condor Workflow is configured using a PHP based gateway that allows users to upload input data and select programs to run TeraGrid Rount Table, October 7, 2010 Source: Archit Kulshrestha, IU

Expressed Sequence Tag Assembly ESTs are a collection of random cDNA sequences, sequenced from a cDNA library or sequencing devices. –Typical inputs are of the order of millions of sequences –Newer 454 devices produce higher volume and are relatively easier to obtain and operate –Stored in a file using the FASTA format The ESTs are clustered and assembled to form contigs. The contigs are then used to identify potential unknown genes, by Blasting against a known protein database. ApplicationPurpose RepeatMaskerCleaning sequences PaCEClustering CAP3Assembly BLASTIdentification Source: Archit Kulshrestha, IU TeraGrid Rount Table, October 7, 2010

Application Runtime Characteristics RepeatMasker Serial Execution on split input Eg for 2 million PaCE MPI – Runtime of several hours Exponential Growth in time with growth in input data. Increasing number of procs works quite well CAP3 Serial Runs on Clusters generated by PaCE – Clusters can be combined Varied sizes with varied resource requirements (run times of milliseconds to days) BLAST Serial – Takes CAP3 results. Number of jobs controlled by adjusting number of sequences per job. Source: Archit Kulshrestha, IU TeraGrid Rount Table, October 7, 2010

Results ProgramNo. Of JobsWait time + Run time Repeat Masker100011:56 PaCE101:22 CAP :44 BLAST89349:00 The results are from a single 2 million job run and hence may not be an accurate model of the wait time. However other than in the case of BLAST the wait times were not a significant component of the total time. Long waits due to long queue times for small jobs. Previous run times – 5 days compared to 2. Serial waits eliminated. Had hooks to inca to determine when jobs were down Failure rate quite low – out of thousands Source: Archit Kulshrestha, IU TeraGrid Rount Table, October 7, 2010

Cyberinfrastructure for Phylogenetic Research (CIPRES) Enables large-scale phylogenetic reconstructions Parallel versions of applications such as MrBayes, Raxml and Garli run on Teragrid Easy to use graphical user interface TeraGrid Rount Table, October 7, 2010

CIPRES Portal users consumed 1,200,000 TeraGrid cpu hours between Dec 2009 and June This was 3 times our projected use. A new award of 2.7 million cpu hours was made on July 1, The portal provides access to parallel versions of MrBayes, RAxML, and GARLI, which all scale well on TG resources. The portal staff has worked with TG special projects group personnel and community developers to provide access to the fastest versions of MrBayes and RAxML available anywhere. Access to BEST, a variant of MrBayes, is planned in the near future. A GPU platform called BEAGLE will be used to provide access to BEAST on Teragrid (Lincoln), also in the near future. The toolkit will be expanded to provide access to other community codes that are appropriate for use on TeraGrid Current Status: Source: Mark Miller, SDSC TeraGrid Rount Table, October 7, 2010

Usage Statistics for CIPRES Portal on TG 12/1/2009 – 5/31/2010 Source: Mark Miller, SDSC TeraGrid Rount Table, October 7, 2010

Intellectual Merit: the CIPRES portal is cited in at least 35 publications this includes publications in Nature, PNAS, and Cell. highlights of scientific findings: New Family Tree for Arthropoda: A team of scientists compared genetic sequences from 75 arthropod species and drew a new family tree for the most successful phylum of animals on Earth. This work represents an important advance in the century-old problem of arthropod evolution. Genome Sequence of a Transitional Eukaryote: A group of scientists sequenced the genome of Naegleria gruberi, a single-cell organism that is a key transitional species between prokaryotes and eukaryotes. This work provides new insights into the origins of subcellular organelles. Co-evolution of Beetles and Flowering Plants: A group of researchers studied the evolutionary history of angiosperms and the beetles that interact with them. The work provided compelling experimental evidence for the long- postulated co-evolution of these two symbiotic groups. Source: Mark Miller, SDSC TeraGrid Rount Table, October 7, 2010

Broad Impacts: 77% of all jobs have been submitted from locations in the USA. Submissions are received regularly from researchers at top-tier institutions such as Harvard, Yale, and Stanford. Jobs are received regularly from academic institutions in 17 EPSCOR states. Job submissions have been received from 34 countries on 5 continents. At least 5 undergraduate classes are known to use the portal routinely. This is likely an underestimate (based on Web log patterns). More than 45,000 jobs have been run on the Portal over its lifetime. Between Dec 1, 2010 and June 30, 2010, users ran 6,108 parallel jobs on the TeraGrid. Source: Mark Miller, SDSC TeraGrid Rount Table, October 7, 2010

Broad Impacts: Impacts on Productivity: Average wall time for RAxML and GARLI jobs decreased 3-4 fold with the shift to TeraGrid resources. Moreover, the number of RAxML jobs has doubled relative to the rate of submission on the CIPRES Portal running on the CIPRES cluster alone. Thus, TeraGrid access is helping users finish their jobs faster and also to make more runs per unit time. The average wall time for MrBayes jobs increased 2-fold on the TeraGrid, but the number of jobs decreased by approximately 33%. This trend reflects users’ ability to run much larger and longer jobs on TeraGrid than on the CIPRES cluster. The increased maximum run-time limit for MrBayes submissions to Abe (168 hours on Abe vs. 72 hours on the CIPRES cluster) allowed users to complete their long runs with a single large submission, thus eliminating the need to make smaller, incremental runs. Source: Mark Miller, SDSC TeraGrid Rount Table, October 7, 2010

Broad Impacts: Improved User Access to TG: 100 – 150 new users per month access TG resources; the number of repeat users is growing…. Source: Mark Miller, SDSC TeraGrid Rount Table, October 7, 2010

New gateway activities in the extension year Helpdesk support expanded –From.2 FTE in PY5 to 1.7 in Extension [NCSA, Purdue] Helpdesk and Condor support, new GIS communities, SimpleGrid extensions Accounting –Improved views for gateways now that we have attributes [TACC] Community accounts –Continued work toward improved standardization [NICS] Prebuilt VMs with gateway software –OGCE, SimpleGrid [IU, NCSA] Online tutorials with CI Tutor and the EOT team –OGCE, SimpleGrid [IU, NCSA] More example-based documentation –Less talk, more action, short videos, based on user feedback [NCSA, SDSC] Remote vis for gateways [ORNL] TeraGrid Rount Table, October 7, 2010

Targeted Support in the Extension All staff available for assignments as new projects come in Cactus –Meet the needs of several groups with large TG allocations [LSU] GridChem, PolarGrid, Ultrascan –Scheduling, vis, Matlab processing, processing of centrifuge data for large international project [IU] CCSM-ESG –Continuing work to combine capabilities [NCAR, Purdue] Uintah, computational fluids [NCAR, Utah] SNS [ORNL] CIPRES [SDSC] OpenSocial for gateways [U Chicago] Improved use of remote vis resources [ORNL] Condor and cloud support [Purdue] TeraGrid Rount Table, October 7, 2010

Gateway Sustainability Study Small, non-TG, EAGER grant Characteristics of short funding cycles –Build exciting prototypes with input from scientists –Work with early adopters to extend capabilities –Tools are publicized, more scientists interested –Funding ends –Scientists who invested their time to use new tools are disillusioned Less likely to try something new again –Start again on new short-term project Need to break this cycle EAGER grant to look at characteristics of successful gateways and domain areas where a gateway could have a big impact TeraGrid Rount Table, October 7, focus group meetings over 2 years First 2 held June,

TeraGrid Rount Table, October 7, 2010 Thank you for your attention! Questions? Nancy Wilkins-Diehr,