Citation and Recognition of contributions using Semantic Provenance Knowledge Captured in the OPeNDAP Software Framework Patrick West 1 James Michaelis 1 Tim Lebo 1 Deborah L. McGuinness 1 Peter Fox 1 ( 1 Rensselaer Polytechnic Institute th St., Troy, NY, United Poster: IN31C-3738 Glossary: OPeNDAP - Open-source Project for a Network Data Access Protocol Provenance – information about entities, activities, and people involved in producing a piece of data or thing, which can be used to form assessments about its quality, reliability or trustworthiness RPI – Rensselaer Polytechnic Institute TWC – Tetherless World Constellation at Rensselaer Polytechnic Institute Sponsors: Tetherless World Constellation Providing proper citation and attribution for published data, derived data products, and the software tools used to generate them, has always been an important aspect of scientific research. However, it is often the case that this type of detailed citation and attribution is lacking. This is in part because it often requires manual markup since dynamic generation of this type of provenance information is not typically done by the tools used to access, manipulate, transform and visualize data. In addition, the tools themselves lack the information needed to be properly cited themselves. The OPeNDAP Hyrax Software Framework is a tool that provides access to and the ability to constrain, manipulate and transform different types of data from different data formats into a common format, the DAP (Data Access Protocol), in order to derive new data products. A user, or another software client, specifies an HTTP URL in order to access a particular piece of data, and appropriately transform it to suit a specific purpose of use. The resulting data products, however, do not contain any information about what data was used to create it, or the software process used to generate it, let alone information that would allow the proper citation and attribution to down stream researchers and tool developers. We will present our approach to provenance capture in Hyrax including a mechanism that can be used to report back to the hosting site any derived products, such as publications and reports, using the W3C PROV recommendation pingback service. We will demonstrate our utilization of Semantic Web and Web standards, the development of an information model that extends the PROV model for provenance capture, and the development of the pingback service. We will present our findings, as well as our practices for providing provenance information, visualization of the provenance information, and the development of pingback services, to better enable scientists and tool developers to be recognized and properly cited for their contributions. Host: opendap.tw.rpi.eduClient: coyote.example.com C: GET S: 200 OK S: Link: rel=“ S: Link: rel=“ (CA_OrangeCo_2011_ ascii representation) Host: opendap.tw.rpi.eduClient: coyote.example.com C: POST HTTP/1.1http://opendap.tw.rpi.edu/disney/pingback C: Content-Type: text/uri-list C: C: C: S: 204 No Content Abstract Proper data management hinges on recording and maintaining “steps” applied to create data. Consumers require methods to assess whether available data is fit for their usage. Was this dataset produced by a trustworthy source? Producers are often expected to justify their efforts in generating new datasets. Who is using our data? What are they using it for? And why? HOWEVER, most current-generation data analysis and manipulation tools fail to capture appropriate meta-information to address these needs. Motivations and Challenges a PROV pingback-enabled community collaborates to categorize the points in a LiDAR scan of Disneyland. A client accesses a data point from a LiDAR scan of Disneyland The client categorizes the point as “water”, which is a new derivation of that point The client pings-back about this new derivation A researcher generates a data product using OPeNDAP and uses it in a derivation. Another researcher, visualizing that derivation, wishes to access the provenance of the data product. What were the original data sources? Can they use them? A scientist wishes to discover any derivations of data sources they created. OPeNDAP.org wishes to be recognized for providing access to, manipulation of, and transformation of derived data products. Use CasesW3C PROV Recommendation Simple Concepts and properties representing the concepts of the PROV Model Representing OPeNDAP provenance trace for use case 1 using PROV Model. OPeNDAP Back-End Server Design :BES_Plan rdf:type prov:Plan, prov:Collection; prov:qualifiedInfluence [ a prov:Influence; prov:entity opendap:NC_Module; prov:hadRole opendap:Read; opendap:order1; ]; prov:qualifiedInfluence [ a prov:Influence; prov:entity opendap:DAP_Module; prov:hadRole opendap:Constrain; opendap:order2; ]; prov:qualifiedInfluence [ a prov:Influence; prov:entity opendap:ASCII_Module; prov:hadRole opendap:Transmit; opendap:order3; ];. :CA_OrangeCo_2011_ nc.ascii rdf:type prov:Entity; prov:wasDerivedFrom :NC_File. prov:wasGeneratedBy :BES_Process;. :BES_Process rdf:type prov:Activity; prov:qualifiedAssociation [ a prov:Association; prov:agent :BES_Agent; prov:hadPlan :BES_Plan; rdfs:comment "Execution of BES ];. :BES_Agent rdf:type prov:Agent; foaf:name "BES Server". RDF Representation of provenance collected in first use case Initial request for dataPingback of derived dataMajor goal to provide visualizations of provenance All the way to seeing who implemented the code that was used to act on the data. Be able to see the software modules that acted on the data in producing the derived data. Initial visualization is of the provenance trace allowing users to get back to the original data, actions taken on the data, and the agents that performed those actions. Take Away With this implementation of provenance capture in OPeNDAP Hyrax we provide data providers and data users the provenance to support their work Providing ping-back service makes it much easier to provide citation and credit to those who deserve it Lots of triples are created, but easily cleaned up by determining what data products have been pinged We can implement provenance capture without modifying existing modules Visualization of provenance trace, agents who act on data products, information about software and software developers becomes easy with linked data principles. Acknowledgements: OPeNDAP.org for their support and being open source, especially James Gallagher and Nathan Potter. First attempt was to capture provenance after-the-fact. Didn’t have enough information in the end. Second attempt is to collect the information on-the-go but not forcing module developers to have to implement anything, but providing hooks to allow them to.