1. Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS* Claudio Gennaro 1, Federica Mandreoli 2,4, Riccardo Martoglia 2, Matteo Mordacchini 1, Wilma Penzo 3,4, and Simona Sassatelli 2 Matteo Mordacchini 1, Wilma Penzo 3,4, and Simona Sassatelli 2 1 ISTI – PI/CNR, Pisa 2 DII - Università degli Studi di Modena e Reggio Emilia 3 DEIS - Università degli Studi di Bologna 4 IEIIT – BO/CNR, Bologna Claudio Gennaro 1, Federica Mandreoli 2,4, Riccardo Martoglia 2, Matteo Mordacchini 1, Wilma Penzo 3,4, and Simona Sassatelli 2 Matteo Mordacchini 1, Wilma Penzo 3,4, and Simona Sassatelli 2 1 ISTI – PI/CNR, Pisa 2 DII - Università degli Studi di Modena e Reggio Emilia 3 DEIS - Università degli Studi di Bologna 4 IEIIT – BO/CNR, Bologna * This work is partially supported by the Italian co-founded Project NeP4B ISDSI 2009 – June 25th, 2009

2. Motivation  ICTs over the Web have become a strategic asset  Internet-based global market place where automatic cooperation and competition are allowed and enhanced  The aim: development of an advanced technological infrastructure for SMEs to allow them to search for partners, exchange data and negotiate without limitations and constraints  The architecture: inspired by Peer Data Management Systems (PDMSs) NeP4B Project (Networked Peers for Business) ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS

3. The Reference Scenario  A peer: a single SME or a mediator  It is queried by exploiting a SPARQL-like query language  similarity predicates (FILTER function LIKE )  It keeps its data in an OWL ontology  Multimedia objects  multimedia attributes in the ontology (e.g. image) Tile price company Image image Peer1 dom ran size dom material dom origin dom ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS SELECT ?company WHERE { ?tile a Tile. ?tile company ?company. ?tile price ?price. ?tile origin ?origin. ?tile image ?image. FILTER ( (?price < 30) && (?origin = ‘Italy’) && LIKE (?image, ‘myimage.jpg’, 0.3))} LIMIT 100

4. The Reference Scenario (cont.)  Peers are connected by means of semantic mapping (with scores)  Peers collaborate in solving users queries How to effectively and efficiently answer a query?  By adopting effective and efficient query routing techniques  Flooding is not adequate Overloads the network (traffic & computational effort) Overwelms the querying peer (irrelevant results) Q Q’ Q’’ Queries are formulated on the peer’s ontology Answers can come from any peer that is connected through a semantic path of mappings ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS

5. Combining Semantic and Multimedia Query Routing Techniques  We leverage our distinct experiences on semantic [Mandreoli et al. WIDM 2006, Mandreoli et al. WISE 2007] and multimedia [Gennaro et al. DBISP2P 2007, Gennaro et al. SAC 2008] query routing  We propose to combine our approaches in order to design an innovative mechanism for a unified data retrieval  We pursue: Effectiveness by selecting the semantically best suited subnetworks Efficiency by promoting the network’s zones where potentially matching objects are more likely to be found, while pruning the others ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS  Two main aspects characterize our scenario: the semantic heterogeneity of the peers’ ontologies the execution of multimedia predicates

6. Outline  Motivation  Query Answering Semantics  Query Routing Semantic Query Routing Multimedia Query Routing Combined Query Routing  Routing Strategies  Experimental Evaluation  Conclusions and Future Works ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS

7. Query Answering Semantics  Peer p i has an ontology O i ={C i1, …, C im }  A semantic mapping: a fuzzy relation M(O i,O j )  O i  O j where each instance (C,C’) has a membership grade μ(C,C’)  [0,1]  evaluation of f on a local data instance i: s(f,i)  [0,1]  s(f,i) = s( f(φ 1, …, φ n ), i ) = sfun φ ( s(φ 1,i), …, s(φ n,i) )  Boolean semantics for relational predicates  non-Boolean semantics for similarity predicates t-norm ( resp. t-conorm) for scoring conjunctions (resp. disjunctions)  Local query answers: Ans(f,p i ) = { (i,s(f,i)) | s(f,i)>0 } Local query execution Query formula: f ::= ::= triple | ∧ ::= φ | ∧ | ∨ | ( ) φ is a relational (=,, =) or similarity (~ t ) predicate pipi OiOi f ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS Ans(f,p i )

8. Query Answering Semantics (cont.)  f undergoes a chain of reformulations f  f 1  …  f m  s(f, P p…pm ) = sfun r (s(f, p 1 ), s(f 1, p 2 ), …, s(f m-1, p m )) sfun r is a t-norm One-step query reformulation (p i  p j )  according to the mapping M(O i,O j )  s(f,p j ) = sfun c ( μ(C 1,C’ 1 ), …, μ(C n,C’ n ) ) sfun c is a t-norm Query answering semantics f submitted to p P’ = {p 1,…,p m } : set of accessed peers P p…pi : path used to reformulate f over each p i in P’  Ans (f, {p} U P’) = Ans(f,p) U Ans(f, P p…p1 ) U … U Ans(f, P p1…pm ) where : Ans(f, P p…pi ) = (Ans(f,p i ), s(f, P p…pi )) Multi-step query reformulation (p  p 1  …  p m = P p1…pm ) pipi pjpj M(O i,O j ) OiOi OjOj f ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS pp1p1 pmpm … f f1f1 f2f2 fmfm (Ans(f,p j ), s(f,p j )) (Ans(f,p m ), s(f,P p…pm ))

9. Outline  Motivation  Query Answering Semantics  Query Routing Semantic Query Routing Multimedia Query Routing Combined Query Routing  Routing Strategies  Experimental Evaluation  Conclusions and Future Works ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS

10. Semantic Query Routing  A Generalized Semantic Mapping relates each each concept C in O i to the set of concepts C s in O j s taken from the mappings in P pi…pjs according to an aggregated score which expresses the semantic similarity between C and C s.  Whenever p i forwards a query to one of its neighbors p j, the query might follow any of the semantic paths originating at p j,, i.e. in p j ’s subnetwork  Main idea: introduction of a ranking approach for query routing which promotes p i ’ neighbors whose subnetworks are the most semantically related to the query. Preliminaries: p j s : the set of peers in the subnetwork rooted at p j O j s : set of schemas {O jk | p jk in p j s } P pi…pjs : set of paths from p i to any peer in p j ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS Peer2 s

11.  When a peer receives a query formula f, it exploits its SRI scores to determine a ranking for its neighborhood: R p sem (f)[p i ] = sfun c (μ(C 1,C 1 s ), …, μ(C n,C n s ))  Each peer p maintains a matrix named Semantic Routing Index (SRI) containing the membership grades given by the generalized semantic mappings between itself and its neighborhood Nb(p) Semantic Query Routing (cont.) SRI Peer1 Tileorigin…Peer11.01.0… Peer20.850.70… Peer30.650.85… ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS  SRI[i,j] represents how the j-th concept is semantically approximated by the subnetwork rooted at the i-th neighbor SRI-based Query Processing

12. Outline  Motivation  Query Answering Semantics  Query Routing Semantic Query Routing Multimedia Query Routing Combined Query Routing  Routing Strategies  Experimental Evaluation  Conclusions and Future Works ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS

13. Multimedia Query Routing  Main idea: introduction of a ranking approach for query routing which promotes p i ’ neighbors whose subnetworks contain the highest number of potentially matching objects. ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS  The execution of multimedia predicates is inherently costly (CPU and I/O) Preliminaries Each peer’s object is classified w.r.t. its distance (dissimilarity) to some reference objects (pivots) E.g.:  object O, pivot P, with d(O, P) = 42 Each peer maintains a condensed description of such a classification of its objects by using histograms (Peer Indices) 0 100 42 0 0 0 0 1  Bit-vector 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 } 0 1 0 0 2

14. ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS Multimedia Query Routing (cont.)  Each peer p also maintains a Multimedia Routing Index (MRI) containing the aggregated description of the resources available in its neighbors’ subnetworks  Similarity-based Range Queries over metrics objects  For each query Q, a vector representation QueryIdx(Q) is built by setting to 1 all the intervals covered by the requested range MRI-based Query Processing R p mm (f)[p i ]= min k [QueryIdx(Q) * MRI(p,p i s )] Σ j=1…n min k [QueryIdx(Q) * MRI (p,p j s )]  When a peer p receives Q, it computes the percentage of potential matching objects in each neighbor’s subnetwork w.r.t the total objects in Nb(p): Any MRI row MRI(p,p i s ) is built by summing up the Peer Indices in the i-th neighbor’s subnetwork MRI Peer1 [0.0,0.2][0.2,0.4]…Peer2829… Peer33001,521…

15. Outline  Motivation  Query Answering Semantics  Query Routing Semantic Query Routing Multimedia Query Routing Combined Query Routing  Routing Strategies  Experimental Evaluation  Conclusions and Future Works ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS

16. Combined Query Routing ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS Optimal aggregation algorithms can only work with monotone aggregation functions  E.g. min, mean, sum…  Both the semantic and multimedia scores induce a total order  They can be combined by means of a meaningful aggregate function in order to obtain a global ranking:  We inspire to Fagin, Lotem & Naor: “Optimal Aggregation Algorithms for Middleware”. Journal of Computer and System Sciences, 66: 47- 58, 2003. R p comb (f) = α R p sem (f)  β R p mm (f)

17. ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS Combined Query Routing (cont.) SRI Peer1 MRI Peer1 min( ) Peer20.700.53 Peer30.650.760.65 E.g. SELECT ?company WHERE { ?tile a Tile. ?tile company ?company. ?tile price ?price. ?tile origin ?origin. ?tile image ?image. FILTER ( (?price < 30) && (?origin = ‘Italy’) && LIKE (?image, ‘myimage.jpg’, 0.3))} LIMIT 100 Final ranking: Peer3-Peer2 Peer3 roots the most promising subnetwork!

18. Outline  Motivation  Query Answering Semantics  Query Routing Semantic Query Routing Multimedia Query Routing Combined Query Routing  Routing Strategies  Experimental Evaluation  Conclusions and Future Works ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS

19. Routing Strategies Efficiency (Depth First Model)  the best peer in one hop! o it exploits the only information provided by R p comb (f) Effectiveness (Global Model)  the best known peer! o it exploits the information provided by U p is visited R p comb (f)  The adopted routing strategy determines the set of visited peers and induces an order on it  When a peer p receives Q it computes the ranking R p comb (f) on its neighbors  Different routing strategies relying on such ranking and having different performance priorities are possible:

20. Outline  Motivation  Query Answering Semantics  Query Routing Semantic Query Routing Multimedia Query Routing Combined Query Routing  Routing Strategies  Experimental Evaluation  Conclusions and Future Works ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS

21. Experimental Evaluation  Simulation environment for SRI and MRI  Peers belong to different semantic categories Ontologies: consisting of a small number of classes Multimedia contents: hundreds of images taken from the Web, characterized by two MPEG7 standard features (scalable color & edge histogram)  Network topology: randomly generated with the BRITE tool in the size of few dozens of nodes  Queries: on randomly selected peers  Routing strategy: DF search  Aggregation function: mean  Stopping condition: number of retrieved results ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS  Queries:  on randomly selected peers  Routing strategy:  DF search  Aggregation function:  mean  Stopping condition:  number of retrieved results  Effectiveness evaluation  We measure the quality of the results (combined satisfaction)  Efficiency evaluation  We measure the number of performed hops Experimental setting

22. Effectiveness Evaluation ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS  We measure the semantic quality of the obtained results (satisfaction)

23. Efficiency Evaluation ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS  We measure the number of hops performed by the query

24. Outline  Motivation  Query Answering Semantics  Query Routing Semantic Query Routing Multimedia Query Routing Combined Query Routing  Routing Strategies  Experimental Evaluation  Conclusions and Future Works ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS

25. Conclusions & Future Works  We presented a novel approach for processing queries effectively and efficiently in a distributed and heterogeneous environment, like the one of the NeP4B Project  We proposed an innovative query routing approach which exploits the semantic of the concepts in the peers’ontologies and the multimedia contents in the peers’repositories  We experimentally prove the effectiveness of our techniques with a series of exploratory tests  (In the future) we will:  perform new tests on larger and more complex scenarios  integrate our techniques in a more general framework for query routing (including latency, costs, etc.) ISDSI’09 - C. Gennaro, F. Mandreoli, R. Martoglia, M. Mordacchini, W. Penzo, S. Sassatell - Combining Semantic and Multimedia Query Routing Techniques for Unified Data Retrieval in a PDMS