1. Development of an Integrated Earth System Climate Modeling Environment
Purdue University/RCAC
NCAR/ESG
NOAA/CIRES

2. A Great Team!
Purdue/RCAC Carol X. Song, Lan Zhao, Chris Thompson, Robert Campbell, Jungha Woo, Han Zhang, Madhavan Lakshminarayanan NCAR Don Middleton, Nathan Wilhelmi, Eric Nienhouse, Michael Burek, Rick Brownrigg NOAA/CIRES Kathy Saint (SGI), Cecelia DeLuca, Sylvia Murphy “Developing an integrated end-to-end TeraGrid climate modeling environment”, TeraGrid 2011 conference, Salt Lake City, UT, July 2011.
05/11/2012
XSEDE Science Gateway

3. Background Motivation for a climate modeling environment Project goals
CCSM/CESM
Resource
Expertise
A collaborative online environment
Existing synergistic community efforts
Earth System Grid, Earth System Curator, ESMF, METAFOR Common Information Model and Purdue CCSM3 portal
Project goals
Provide an integrated environment for climate modeling, data/metadata archival, visualization and analysis
Support research and education, lower barriers to using climate models
05/11/2012
XSEDE Science Gateway

4. Authentication/ Authorization
System Architecture
CESM portal
ESG gateway
Account DB
User
requests
TG MyProxy
Token Mgr
CESM Web Services
SOAP and Restful XML/JSON
Create Case
Configure Case
Submit Case
Track Status
Data/
Metadata
Authentication/ Authorization
Transfer Files
Job Management
Debugging
Post-process
Publish Data
Publish Metadata
XD community account/Individual account
Output
Output
Jobs
ESG Data Publisher
Scratch Storage
iRODS
05/11/2012
XSEDE Science Gateway

5. CESM Portal - Model Workflow
Insulate users from cluster
Hide command line details
Integrate CESM metadata capabilities + ESG publishing + NCL
Web services (SOAP, Restful XML/JSON)
Manage user’s experience
Maintain job status and metrics
End-to-end modeling from simulation to data/metadata publishing
Bind independent processes into unified workflow
CESM Portal
JBOSS
Job Scripts
Publish Scripts
SOAP
CESM-WS/ gateway
Create case
CESM Model
Create
create_case
Configure case
Configure
./configure
Compile
<case>.compile
Compile case
Execute
<case>.submit
Query Status
Execute case
Data
Configure post-process
Post-processing
Post-processing Engines
Run post-process
AMWG
NCL scripts
ESG Publication
workflow …
05/11/2012
XSEDE Science Gateway

6. ESMF Attributes and Self-describing Models
Metadata is increasingly important in Earth science modeling
Understand multi-component models with many options and parameters
Distinguish simulations from one another
Archive simulations with full descriptions for preservation and defensibility
Enable run reproducibility
ESMF Attributes store, aggregate, and output metadata
Structured as name-value pairs
Arranged in packages by purpose and convention: International Standards Organization (ISO), Climate and Forecast (CF), METAFOR Common Information Model (CIM), etc.
Output in XML and other formats
ESMF Attributes in CESM
CESM components (land, atm, ice, ocean, sea ice) and driver are being instrumented with Attributes
Information captured includes component level, field level, and system level metadata
05/11/2012
XSEDE Science Gateway

7. CESM Metadata in the ESG Gateway
Screenshot of CESM metadata
Implemented in the model by ESMF and Curator.
Exported by CESM at run time.
Uploaded to Purdue’s atom feed.
Ingested and displayed by ESG.
05/11/2012
XSEDE Science Gateway

8. Data Discovery, Access and Analysis
ESG: Science Gateway for Earth System Data Access
Data discovery and access across federation of data nodes.
Identity services and project specific access control.
Data publishing services.
Versioning and provenance support.
Bulk data download, wget scripts.
Data use tracking and metrics.
Model documentation metadata browsing.
ATOM CIM feed and ESG ingest.
NCL Backend to Product Services
Integrated 64-bit NCL visualization engine.
Big data capable analysis functions.
ParNCL as a drop-in replacement, for parallel computation.
Plug-in component to NOAA/PMEL LAS Product Server.
05/11/2012
XSEDE Science Gateway

9. Demo
05/11/2012
XSEDE Science Gateway

10. Demo
05/11/2012
XSEDE Science Gateway

11. Demo
05/11/2012
XSEDE Science Gateway

12. Next Steps Transition into XD
Release CESM portal
Work with faculty and instructors to use in teaching
Need help in connecting with users
Federated authentication/identity between CESM and ESG gateways
Making a rich set of tools for analysis and visualization available to CESM gateway users
Add other resources to backend for better user experience
05/11/2012
XSEDE Science Gateway