Efrat Jaeger – SDSC Bertram Ludäscher – UC DAVIS Krishna Sinha – Virginia Tech Ashraf Memon – SDSC Ghulam Memon – SDSC Ilkay Altintas – SDSC Kai Lin – SDSC & many others esp. KEPLER community San Diego Supercomputer Center UC DAVIS Department of Computer Science CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Scientific Workflows & GEON

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Scientific Workflows Pre-Cyberinfrastructure Data Federation & Grid “Plumbing”: – access, move, replicate, query … data (Data-Grid) authenticate … SRB Sget/Sput … OPeNDAP, … Antelope/ORBs – schedule, launch, monitor jobs (Compute-Grid) Globus, Condor, Nimrod, APST, … Data Integration: – Conceptual querying & integration, structure & semantics, e.g. mediation w/ SQL, XQuery + OWL (Semantics-enabled Mediator) Data Analysis, Mining, Knowledge Discovery: – manual/textbook (e.g. ternary diagrams), Excel, R, simulations, … Visualization: – 3-D (volume), 4-D (spatio-temporal), n-D (conceptual views) …  one-of-a-kind custom apps., detached (island) solutions  workflows are hard to reproduce, maintain  no/little workflow design, automation, reuse, documentation  need for an integrated scientific workflow environment

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Analysis Workflow in KEPLER Scientific Workflow (SWF) design SWF automation Exploration & discovery mode (change parameters, data sets, etc. and rerun) SWF reuse, documentation, reproducibility

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Some KEPLER Components (Actors)

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES KEPLER Team Work: GEON Dataset Generation & Registration Xiaowen (SDM) Edward et al.(Ptolemy) Yang (Ptolemy) Efrat (GEON) Ilkay (SDM) SQL database access (JDBC) Matt,Chad, Dan et al. (SEEK) % Makefile $> ant run % Makefile $> ant run

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES KEPLER: an open source, cross-project collaboration Ilkay Altintas SDM, Resurgence, NLADR,… Kim Baldridge Resurgence, NMI Chad Berkley SEEK Shawn Bowers SEEK Terence Critchlow SDM Tobin Fricke ROADNet Jeffrey Grethe BIRN Christopher H. Brooks Ptolemy II Zhengang Cheng SDM Dan Higgins SEEK Efrat Jaeger GEON Matt Jones SEEK Werner Krebs, EOL Edward A. Lee Ptolemy II Kai Lin GEON Bertram Ludaescher GEON, SDM, SEEK, BIRN, ROADNet Mark Miller EOL Steve Mock NMI Steve Neuendorffer Ptolemy II Jing Tao SEEK Mladen Vouk SDM Xiaowen Xin SDM Yang Zhao Ptolemy II Bing Zhu SEEK Ptolemy II Your Logos & Names HERE!!!

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Demonstration by Efrat Jaeger

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Q & A

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES KEPLER: An Open Collaboration Initiated by members from NSF/ITR SEEK and DOE SDM/SPA; now several other projects (GEON, Ptolemy II, EOL, Resurgence/NMI, …) Open Source (BSD-style license) Intensive Communications: – Web-archived mailing lists – IRC (!) – Meetings, Hackathons Co-development: – via shared CVS repository – joining as a new co-developer (currently): get a CVS account (read-only) local development + contribution via existing KEPLER member be voted “in” as a member/co-developer

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Scientific Workflow (SWF) Design Support SWF design & reuse, via: – Structural data types – Semantic types – Associations (=constraints) between them – Type checking, inference, propagation  Separation of concerns: – structure, semantics, WF orchestration, etc.

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Related Publications Scientific Workflows Scientific Workflow Management and the Kepler System, B. Ludäscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger-Frank, M. Jones, E. Lee, J. Tao, Y. Zhao, Concurrency and Computation: Practice & Experience, Special Issue on Scientific Workflows, to appear, Scientific Workflow Management and the Kepler System A Framework for the Design and Reuse of Grid Workflows, Ilkay Altintas, Adam Birnbaum, Kim Baldridge, Wibke Sudholt, Mark Miller, Celine Amoreira, Yohann Potier, and Bertram Ludaescher, Intl. Workshop on Scientific Applications on Grid Computing (SAG'04), LNCS 3458, Springer, 2005 A Framework for the Design and Reuse of Grid WorkflowsSAG'04 Kepler: An Extensible System for Design and Execution of Scientific Workflows, I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludäscher, S. Mock, 16th International Conference on Scientific and Statistical Database Management (SSDBM'04), June 2004, Santorini Island, Greece. Kepler: An Extensible System for Design and Execution of Scientific WorkflowsSSDBM'04 Kepler: Towards a Grid-Enabled System for Scientific Workflows, Ilkay Altintas, Chad Berkley, Efrat Jaeger, Matthew Jones, Bertram Ludäscher, Steve Mock, Workflow in Grid Systems (GGF10), Berlin, March 9th, Kepler: Towards a Grid-Enabled System for Scientific WorkflowsWorkflow in Grid Systems (GGF10) An Ontology-Driven Framework for Data Transformation in Scientific Workflows, S. Bowers and B. Ludäscher, Intl. Workshop on Data Integration in the Life Sciences (DILS'04), March 25-26, 2004 Leipzig, Germany, LNCS An Ontology-Driven Framework for Data Transformation in Scientific WorkflowsDILS'04 A Web Service Composition and Deployment Framework for Scientific Workflows, I. Altintas, E. Jaeger, K. Lin, B. Ludaescher, A. Memon, In the 2nd Intl. Conference on Web Services (ICWS), San Diego, California, July 2004.ICWS

GEON AHM May 5-6, 2005, San Diego CYBERINFRASTRUCTURE FOR THE GEOSCIENCES DataIntegration KnowledgeRepresentation Process Integration (Scientific Workflows) Source: B. Ludaescher, UC DAVIS ECS-289 Scientific Data Management WQ’05 DataFederation EcoGrid