1. An RDF and XML Database John Snelson, Lead Engineer 23 rd October 2013

2. Slide 2 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. MarkLogic SEARCHDATABASE APPLICATION SERVICES

3. Slide 3 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Data ≠ Information

4. Slide 4 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Data + Context = Information

5. Slide 5 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Dynamic Semantic Publishing BBC Sports Size and Complexity:  # of athletes  # of teams  # of assets (match reports, statistics, etc.)  # of relations (facts)  Rich user experience  See information in context  Personalize content  Easy navigation  Intelligently serve ads (outside of UK)  Manageable  Static pages? Too many, changing too fast  Limited number of journalists  Automate as much as possible The ChallengeGoals

6. Slide 6 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Dynamic Semantic Publishing A Solution  Store, manage documents  Stories  Blogs  Feeds  Profiles  Store, manage values  Statistics  Full-Text search  Performance, scalability  Robustness  Metadata about documents  Tagged by journalists  Added (semi- )automatically  Inferred  Facts reported by journalists  Linked Open Data for real-world facts XML DatabaseTriple Store

7. Slide 7 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. played in plays in plays for Dynamic Semantic Publishing Understanding Data

8. Slide 8 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Dynamic Semantic Publishing Scaling Up

9. Slide 9 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. What is RDF? :has-child :has-parent :birth-place :spouse :birth-place :has-child :has-parent :person20 :person5 :place5 :first-name :person4 “John”

10. Slide 10 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. What is RDF? Schema-less Triple granularity Open world assumption Joins - the cost of granularity RDF

11. Slide 11 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Data stored in Triples Expressed as Subject : Predicate : Object Example: "John Smith" : livesIn : "London" "London" : isIn : "England" What is Semantics?

12. Slide 12 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Data stored in Triples Expressed as Subject : Predicate : Object Example: "John Smith" : livesIn : "London" "London" : isIn : "England" Rules tell us something about the triples Example: If (A livesIn X) AND (X isIn Y) then (A livesIn Y) Inference: "John Smith" : livesIn : "England" What is Semantics?

13. Slide 13 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Data stored in Triples Expressed as Subject : Predicate : Object Example: "John Smith" : livesIn : "London" "London" : isIn : "England" Rules tell us something about the triples What is Semantics? "John Smith" "England" livesIn "London" isIn livesIn

14. Slide 14 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Why use RDF? Born or extracted to RDF Denormalize into XML by default Lift data into RDF if you need to: combine it with disparate data sources navigate it like a graph use it for relationships or taxonomy expose it as RDF to end users RDF

15. Slide 15 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Semantics Architecture TRIPLE XQY XSLT SQLSPARQL GRAPH SPARQL

16. Slide 16 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Triple Index 3 triple orders Cached for performance Works seamlessly with other indexes Security 150 bytes per triple on disk Billions of triples per host Scaling out horizontally TRIPLE

17. Slide 17 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. RDF Loading RDF

18. Slide 18 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Triples Embedded in Documents … http://example.org/kennedy/person12 http://example.org/kennedy/last-name <sem:object datatype="http://www.w3.org/2001/XMLSchema#string"> Lawford …

19. Slide 19 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Content, Data, and Semantics Suspicious vehicle… Suspicious vehicle near airport 2012-11-12Z observation/surveillance suspicious activity suspicious vehicle 37.497075 -122.363319 IRIID IRIID isa value license-plate ABC 123 A blue van… A blue van with license plate ABC 123 was observed parked behind the airport sign…

20. Slide 20 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Content, Data, and Semantics Suspicious vehicle… 2012-11-12Z suspicious activity suspicious vehicle 37.497075 -122.363319 A blue van… IRIID isa value license-plate ABC 123 observation/surveillance Semant ic ( RDF ) Triple s Unstructure d full - text Geosp atial Dat a

21. Slide 21 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. RDF Values “string value”^^xs:string “987”^^xs:double “2013-04-09”^^xs:date “bonjour”@fr _:blank1 “simple”

22. Slide 22 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Datatype Mapping DatatypeSPARQLXQuery Typed Literal “2013-04-09”^^xs:datexs:date(“2013-04-09”) IRI sem:iri(“http:// example.com”) Blank Node _:blank1 sem:blank(“…”) Simple Literal “simple”xs:string(“simple”) Language “bonjour”@fr Tagged Literal rdf:langString(“bonjour”, “fr”)

23. Slide 23 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. SPARQL Executed using the triple index SPARQL 1.0 + much of SPARQL 1.1 Cost-based optimization Join ordering and algorithms select * where { ?person :birth-place ?place; :first-name “John” } SPARQL

24. Slide 24 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Executing SPARQL sem:sparql(“ prefix : select * { ?person :first-name ?first; :last-name ?last; :alma-mater [:ivy-league :true] }”, map:entry(“first”,“John”), (), cts:collection-query(“mycollection”) )

25. Slide 25 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Returning Binding Solutions select * where { ?person :birth-place :place5 } select * where { ?person :birth-place ?place; :first-name “John” }

26. Slide 26 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Solution Results personplace :person22:place13 :person4:place5

27. Slide 27 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. SPARQL Query Results XML Format sem:query-result-serialize( sem:sparql(“select * { … }”), “xml” )

28. Slide 28 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Returning Triples describe :person4 construct { ?bp :uses-name ?fn } where { ?person :birth-place ?bp; :first-name ?fn }

29. Slide 29 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Triple Results :place0 :uses-name “Ethel”, “Jeffrey”, “Kara”. :place1 :uses-name “Edward”, “James”. :place10 :uses-name “Robert”, “Sheila”, “Stephen”.

30. Slide 30 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Querying Named Graphs select * from where { ?s ?p ?o } select * where { graph { ?s ?p ?o }

31. Slide 31 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Restricting The Datasets let $options := “properties” let $query := cts:and-query( cts:directory-query(“/triples/”), cts:element-range-query( xs:QName(“date”),“>”,$date) ) return sem:sparql(“…”,(),(), $options,$query)

32. Slide 32 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Creating Triples sem:triple() sem:rdf-parse() sem:rdf-get() sem:rdf-builder() sem:rdf-load() sem:rdf-insert() Returning sem:triple valuesInserting to a database

33. Slide 33 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Graph Store API declare function graph-insert( $graphname as sem:iri, $triples as sem:triple*, [$permissions as element(sec:permission)*, $collections as xs:string*, $quality as xs:int?, $forest-ids as xs:unsignedLong*] ) as xs:string*; declare function graph-delete( $graphname as sem:iri ) as empty-sequence();

34. Slide 34 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Conclusion Semantics can enhance your data- oriented and search applications. XQuery and SPARQL work well together. A combination RDF and XML database simplifies working with the technologies together. Try MarkLogic 7: http://www.marklogic.com/early-access/ http://www.marklogic.com/early-access/

35. Slide 35 Copyright © 2013 MarkLogic ® Corporation. All rights reserved. Any Questions?