Presentation is loading. Please wait.

Presentation is loading. Please wait.

Triplestore Experiences Nathan Wilhelmi 11/27/2012 NCAR - CISL/TDD/VETS.

Similar presentations


Presentation on theme: "Triplestore Experiences Nathan Wilhelmi 11/27/2012 NCAR - CISL/TDD/VETS."— Presentation transcript:

1 Triplestore Experiences Nathan Wilhelmi 11/27/2012 NCAR - CISL/TDD/VETS

2 Our Experiences… Disclaimers: ▫Did not have an ontologist ▫Codebase passed through multiple developers Timelines (changing landscape) ▫Started work 2006 ▫Stopped active development ~2011 ▫Sesame version 2.3.0

3 Why a Triplestore? Search functionality ▫Faceted ▫Free text Model metadata ▫Metadata storage ▫Display Semantic web

4 Initial Architecture Authoritative metadata source was RDBMS Metadata harvested into the triplestore at periodic intervals Triplestore only contained metadata to drive search Sesame used as a stand alone service

5 Sesame Triplestore Standalone Sesame server ▫Stability problems ▫No security, triplestore could be updated by anyone Changed to in-memory store ▫Stable ▫Picked up performance improvements Embedded triplestore was only internally referencing ▫RDF didn’t work outside of the application ▫Distilled to key-value store

6 Internal Referencing

7 Performance For our query patterns were not seeing needed performance Inferencing was removed and performance improved to acceptable levels for <5k datasets ▫ Target volume 50K datasets Sparql missing key operators: ordering, limits

8 Tooling Support Managing the triplestore ▫Protégé round trips didn’t work well ▫Dump full triple store to XML and grep by hand Deleting and updating triples ▫Deletes were difficult, dangling triples ▫Rebuild from authoritative sources

9 Implementation Issues Schema-less design was perceived as faster ▫Rapid ontology changes during development ▫Still needed data migration tools Modeling the problem domain ▫Modeled a triplestore, not the domain ▫Very tightly coupled code was difficult to maintain and replace Steep learning curve for new developers

10

11 URIs Are Foundational Properly encoding URIs ▫Created unencoded URIs within the triplestore ▫Queries were created with string concentration ▫Lead to broken queries and data Generated instance URIs through a lossy algorithm to get around encoding ▫Could only relate from source -> triple store

12 Our Current Path Forward Using SOLR Search ▫Fantastic search tool! Metadata in RDBMS ▫Working well ▫Effective tools, including schema migration ▫Scales very well for our metadata Still needed to expose RDF metadata…

13 RDF with RDBMS resource://${gateway.name?upper_case}#${dataset.persistentIdentifier} ${dataset.name}

14 Looking Forward Storing metadata ▫Content management systems? ▫NoSql storage options? Modeling complicated relationships ▫Neo4J looks promising…

15 Questions / Discussion Nathan Wilhelmi


Download ppt "Triplestore Experiences Nathan Wilhelmi 11/27/2012 NCAR - CISL/TDD/VETS."

Similar presentations


Ads by Google