HydroShare: An online collaborative environment for the sharing of hydrologic data and models IN11A-1510 We envision that HydroShare will enable more rapid advances in hydrologic understanding through collaborative data sharing, analysis, and modeling. HydroShare will provide a community collaboration site that enables users to easily discover and access data and models, retrieve them to a desktop computer or perform analyses in a distributed computing environment that includes grid, cloud, or high performance computing model instances as necessary. Understanding will be advanced through the ability to integrate information from multiple sources. Outcomes (data, results, models) can then be published as new resources that can be shared with collaborators. Our goal is to make sharing of hydrologic data and models as easy as sharing videos on YouTube or shopping on Amazon. OCI OCI David G. Tarboton, Ray Idaszak, Jeffery S. Horsburgh, Daniel Ames, Jonathan L. Goodall, Lawrence E. Band, Venkatesh Merwade, Alva Couch, Jennifer Arrigo, Richard P. Hooper, David W. Valentine, David R. Maidment 1 Observers and instruments Data Analysis Models Collaboration Publication, Archival, Curation Data are observed and then loaded into CUAHSI HydroServer. 2.HydroServer stores data in a relational database and publishes it using web services. Metadata is harvested and supports geographic and context based data discovery. 3.A HydroDesktop client user discovers, downloads and analyzes the data, or uses it in a model. 4.The user next posts the results (data and model) to HydroShare as resources (social objects) retaining provenance information on the original data source. Upon ingestion background actions (microservices) parse metadata and enable analysis based on rules and policies. 5.The user shares posted resources with colleagues (access controlled). Group collaboration and analysis on refining the analysis, model or result. 6.After iteration the result is finalized and submitted for publication. 7.Upon publication the resources produced (data, model, workflow, paper) are made immutable, access is opened and permanent persistent identifiers assigned. Collaborative data analysis and publication use case Collaborative integrated modeling use case 1.Data: Links to national and global data sets of essential terrestrial variables (e.g. NASA NEX, HydroTerre) 2.Tools to preprocess and configure inputs (EcoHydroLib) 3.Preconfigured models and modeling systems as services (SwatShare) 4.Standards for information exchange for interoperability (OpenMI, CSDMS BMI, DataNet Federation Consortium, DataOne) 5.Tools for visualization and analysis 6.Automated reasoning to couple models based on purpose, context, data and resources Flow Time P Analysis Tools Visualization Tools Data Loaders Data Discovery Tools Models Resource Repository Resource Centric Paradigm for Modeling and Analysis Enable multiple models to use common “best practice” tools Each model interacts with information in the common data store The modeler does not need to be concerned with and can take advantage of standardized analysis, visualization loading and discovery tools x y t Flow Time Pre-processing and model linking Modeling Services (e.g. SWATShare) Contact: Learn more: Web Browser Drupal iRODS HydroDesktop Indexing Data Grid RESTful Interface Policy Enforcement Point Implemented Coming Soon Computation Workflow Structured Objects GeoAnalyticsFramework Architecture and Implementation Drupal Content Management System Extensible Open Source Content Management Framework for Publication written in PHP – Over 14,000 user contributed modules Themed and Styled Presentation of HydroShare Resources with in page visualization Off the shelf modules provide a Social Experience surrounding Hydrologic Data: Comments, Ratings, Group Behavior Custom module development supports HydroShare Data Model, GeoAnalytics and iRODS Integration iRODS Storage of HydroShare Resources Replicated across multiple institutions Rule engine for policy driven data management Extensibility via microservices Collaborative open source open development using GitHUB and Codeplex HydroShare is currently in beta testing. The HydroShare Beta Site located at is open for you to use, recognizing the sometimes buggy nature of in development beta systems. So please keep backups of any content in beta.hydroshare.org that is important to you. Please provide us suggestion or feedback on problems you experience so we can improve HydroShare. Your participation as a beta tester is greatly appreciated. Vision and Goal Resource Data Model HydroShare Resource Resource Map Aggregation Aggregated Object 1.. * * Open Archives Initiative – Object Reuse and Exchange (OAI-ORE) - defines standards for the description and exchange of aggregations of Web resources BagIt – hierarchical file packaging format designed to support disk-based or network-based storage and transfer of generalized digital content Data and metadata files Example 1: Time Series as WaterML Resource Map Science Metadata WaterML file Example 2: Hydrologic Model Resource map Science metadata file Model executable file Source folder – Many source files Resource Map Science Metadata Science Metadata source3.f90 Model.exe sourcen.f90 source1.f90 source2.f90... source Resource Map Science Metadata Science Metadata WaterML Resource Types Time Series Geographic feature set Other Referenced HIS time series Geographic raster Multidimensional space time dataset River Geometry Sample based observations (ODM2 and CZO) Document Tabular objects HydroDesktop project package Scripts Models Model component Referenced data set from web service Creator Contributor Coverage Title Date Description Type Format Identifier Language Publisher Relation Rights Source Subject Created Modified Issued Valid Name Organization Mail address Phone Attributes common to all resources Spatial Temporal siteInfo VariableInfo QualityControlLevel Method Source seriesInfo Attributes of time series (WaterML) Child attributes … Spatial Coverage Temporal Coverage Model Type Instance Component Attributes of Model Begin Date&Time End Date&Time Temporal Interval Instance Component Parent Model Version Description URL Modeling Framework Version Description URL All content is represented using the Resource Data Model that separates system and science metadata and has elements common to all resources as well as elements specific to the types of resources HydroShare will support. Specifying a resource data type requires Data content and logical relationships Name and type of all data and metadata elements Indicate whether elements are required or optional Indicate whether elements are from a vocabulary File formats selection or definition Spatial Resolution Reference System Domain