1. Finding Hidden Correlations and Filtering out Incorrect Matchings with Compatibility Detection across Web Query Interfaces Lei Lei June 11, 2004 June 11, 2004

2. Introduction Deep Web scales rapidly Deep Web scales rapidly Proliferating sources with structured Info. Proliferating sources with structured Info. Vocabulary Converge to small size Vocabulary Converge to small size Dynamic Queries instead of URLs Dynamic Queries instead of URLs

3. Complex Matching Traditional methods focus on 1:1 matching Traditional methods focus on 1:1 matching Query shemas form Complex Matchings Query shemas form Complex Matchings M:n M:n

4. Web Query Interfaces Web Query Interfaces Web Query Interfaces Attribute Group

5. Problems to solve Relations are complicated and multi-ary Relations are complicated and multi-ary How to Judge the Relations of Synonyms? How to Judge the Relations of Synonyms? How to pick out incorrect matchings? How to pick out incorrect matchings?

6. Statement Find out the hidden synonyms and build correlations to solve m:n matching problem Find out the hidden synonyms and build correlations to solve m:n matching problem Filter out false matchings and partially incorrect ones with the three step “ compatibility detection ”. Filter out false matchings and partially incorrect ones with the three step “ compatibility detection ”.

7. MGSsd and Improved Model Original Hidden Model from MGSsd Original Hidden Model from MGSsd

8. Find Hidden Synonyms Assume existence of hidden synonyms Assume existence of hidden synonyms Correlations between synonyms Correlations between synonyms Function: HC(bi,bj) Function: HC(bi,bj) Apply HC directly Apply HC directly

9. Example Synonyms on air booking domain Synonyms on air booking domain Set a Threshold Set a Threshold HC (b2,b4)

10. Compatibility Detection Not all raw matching are correct Not all raw matching are correct Clean partially correct or inaccurate ones Clean partially correct or inaccurate ones Three Steps: Three Steps: Transitivity Check Transitivity Check Examine Confidence Examine Confidence Subsumption Subsumption

11. Compatibility Detection(Cont.) Raw Matching Results Raw Matching Results 1.Check Transitivity 1.Check Transitivity 2. Choose Confidence 3. Subsumption

12. Evaluation Using Recall and Precision Using Recall and Precision Compare with MSGsd data Compare with MSGsd data Perform Correlation and Compatibility on matching results from other researches Perform Correlation and Compatibility on matching results from other researches

13. Contributions m:n mapping rather than only 1:1 mapping m:n mapping rather than only 1:1 mapping Present a hidden synonym approach to statistically compute the correlation between synonym groups Present a hidden synonym approach to statistically compute the correlation between synonym groups Develop the “ Compatibility Detection ” approach to refine the raw mapping data Develop the “ Compatibility Detection ” approach to refine the raw mapping data Suitable and efficient as the Web scales Suitable and efficient as the Web scales

14. Future Work Figure out the HC Function Figure out the HC Function “ Minimum ” is feasible “ Minimum ” is feasible Distinguish Trivial Difference in Confidence Distinguish Trivial Difference in Confidence Set up a proper threshold Set up a proper threshold Space Complexity Space Complexity Type Subsumption Type Subsumption Departing: datetime Departing: datetime Departing: string Departing: string

15. Questions ? Questions ?