The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 1 MICS Scientific Conference Message-Passing in Peer Data Management Systems Philippe Cudré-Mauroux and Karl Aberer School of Computer and Communication Sciences EPFL -- Switzerland

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 2 Motivation: Sharing Content in Large-Scale Systems Profusion of Digital Images –Variety of powerful devices –gigabytes of pictures is the new norm Most of the images are kept local Some are shared –Mostly point-to-point –Primitive search capabilities  keyword search MMS HTTP SMTP

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 3 Opportunity More and more software use structured metadata to organize images locally –Structured metadata (e.g., XML, PSA) –Ontological metadata (e.g., RDF, XMP) –Type-based metadata (e.g., WinFS) <rdf:RDF xmlns:rdf= ' T18:49:03Z T20:09:28Z John Doe …

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 4 Hurdle: Metadata Heterogeneity Why not taking advantage of those metadata in a distributed setting? Problem: autonomy of sources X Syntactic discrepancies X Semantic heterogeneity All the aforementioned standards are extensible Shared representation is not enough ImageGUIDcDate A0657B E7A /08/2004 VS Width Length-Y VS

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 5 Decentralized Semantics Traditionally in databases one centralized representation (schema) is used to integrate data sources Not applicable to our context –centralized violates autonomy How can we foster semantic interoperability in decentralized settings? Date myDate yourDate m(Date) = yourDate m(Date) = myDate

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 6 Beyond Keyword Search  searching semantically richer objects in overlay networks T18:49:03Z T20:09:28Z date? 05/08/2004 Jan 1, 2005 ? ? ? ? ? ?

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 7 Semantic Interoperability Q1= $p/GUID FOR $p IN /Photoshop_Image WHERE $p/Creator LIKE "%Robi%" 178A8CD8865 Robinson Tunbridge Wells Royal Council … Photoshop (own schema) 178A8CD8866 Henry Peach Robinson Photographer Tunbridge Council … WinFS (known schema ) T12 = $fs/GUID $fs/Author/DisplayName FOR $fs IN /WinFSImage Q2= $p/GUID FOR $p IN T12 WHERE $p/Creator LIKE "%Robi%"  Extending semantic interoperability techniques to decentralized settings!

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 8 Pairwise Schema Mappings Local mappings enabling global agreement –Peer Data Management Systems (PDMS) Pairwise mappings overcome schema heterogeneity –Transitive closures on mapping operations –Mappings can be redundant T18:49:03Z T20:09:28Z date? 05/08/2004 Jan 1, 2005 article weather es:cDate  xap:CreateDate es:cDate  myRDF:Date myRDF:Date  xap:ModifyDate

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 9 Problem: Optimizing Precision/Recall of Answers Semantic Query routing –To whom shall I forward a query posed against my local schema? Some (most) mappings will be (partially) faulty –Low expressive power of mappings –Automatic schema alignment techniques –Granularity of conceptualizations… Local query resolution –Low recall Flooding –Low precision Standard deductive integration is not sufficient –Uncertainty on mappings and conceptualizations  abductive reasoning (on transitive closures of mappings)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 10 On Mapping Cycles / Parallel Paths m0m0 m1m1 m2m2 m3m3 m4m4 m5m5

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 11 Computing a Marginal for one cycle P(m 0, m 1, m 2, m 3, f 0 ) = P(m 0 ) P(m 1 ) P(m 2 ) P(m 3 ) P(f 0 | m 0, m 1, m 2, m 3, ) P(m 0 | f 0 )=  m1, m2, m3 P(m 0, m 1, m 2, m 3, f 0 ) P(f 0 ) -1 But: feedback on different cycles are correlated –Need to express a global probabilistic model for the mapping graph observedunknown

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 12 A Brief Intro to Factor-Graphs g(x1, x2, x3, x4) = fA(x1, x2)fB(x2, x3, x4)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 13 Deriving PDMS Factor-Graphs

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 14 PDMS Factor-Graphs Cyclic graph –Junction Tree? Clustering / Stretching of variables? Not applicable (decentralization) –Iterative Sum-Product Approximate results How to perform iterative sum-product by message passing on the mapping graph? –Message passing in factor graph does not correspond to connectivity of mapping graph –We want to rely on decentralized computations only Locality VS Globality of nodes in the factor graph –Mappings: local –Feedback factor: common, global knowledge –Observed feedback variables: neighborhood

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 15 Embedded Message-Passing (1)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 16 Embedded Message-Passing (2)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 17 Sending Messages in the Mapping Graph Message-Passing Schedules –Periodic –Lazy (piggybacking on query forwarding)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 18 Implemented System Schemas –Import from OWL (Web Ontology Language) Mappings –KnowledgeWeb Ontology Alignment API –Import from RDF/XML –Automated on-the-fly creation –Comparison to standard alignments  Automatic derivation of quality measures P(m=correct | {F}) for the mappings using iterative message-passing  Per-Hop Forwarding Behaviors (Semantic Gossiping)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 19 Some (Preliminary) Results: Convergence (undirected example graph, prior 0.7 delta 0.1)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 20 Impact Of Cycle Length (simple cycle, prior 0.5)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 21 Fault-tolerance (faulty links) (undirected example graph, prior 0.8 delta 0.1)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 22 Preliminary Results: EON schemas Worst-case scenario: no prior knowledge Set of 6 schemas on bibliographic data (approx attributes) 396 generated attribute mappings (84 incorrect)

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 23 Semantic Mediation Layer Correlated / Uncorrelated Correlated / Uncorrelated “Physical” layer Overlay Layer Semantic Mediation Layer

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 24 An Implemented System GridVine

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 25 Conclusions and Future Works Abduction on closures of mappings to automatically derive mapping quality measures in PDMS Totally decentralized solutions –Scalable Approximate (but practically useful) results Future works –Other inference techniques Decentralization –Prior Belief Updates –Dynamics of Semantic Networks under churn Mapping Graph  Global Semantic Interoperability

The National Centres of Competence in Research are managed by the Swiss National Science Foundation on behalf of the Federal Authorities 26 References Message-Passing in Peer-Data Management Systems Philippe Cudré-Mauroux, Karl Aberer, and Andras Feher Accepted for publication, 22nd International Conference on Data Engineering (ICDE2006) Semantic Overlay Netwoks (tutorial) Karl Aberer and Philippe Cudré-Mauroux Thirty-first International Conference on Very Large Data Bases (VLDB 05) GridVine: Building Internet-Scale Semantic Overlay Networks Karl Aberer, Philippe Cudré-Mauroux, Manfred Hauswirth, and Tim van Pelt Third International Semantic Web Conference (ISWC04) The Chatty Web: Emergent Semantics through Gossiping Karl Aberer, Philippe Cudré-Mauroux, and Manfred Hauswirth, Twelfth International World Wide Web Conference (WWW03) A Framework for Semantic Gossiping Karl Aberer, Philippe Cudré-Mauroux, and Manfred Hauswirth SIGMOD Record, 31(4), December 2002