1. 1 Approximate Data Exchange Michel de Rougemont Adrien Vieilleribière University Paris II & LRI University Paris-Sud & LRI ICDT 2007

2. 2 1.Data from different imperfect sources. Framework for Data-Exchange and Data- Integration 2.Logic and Approximation Definability and Complexity (scaling) Robustness 3.Statistics based computations Motivation

3. 3 1.Classical Data Exchange on words and trees 2.Approximation based on Property Testing. Tester for regular words and regular trees (Edit Distance with Moves) Property testing for regular tree languages (ICALP 2004) Approximate Satisfiability and Equivalence (LICS 06) 3.Approximate Data Exchange Plan

4. 4 1. Data Exchange on Trees Source Targets ?

5. 5 Data Exchange setting: (K S,τ,K T ) Fagin et al. 2002: τ defined by Source-Target-Dependencies on relations Arenas, Libkin 2005: τ defined by Tree-Pattern-Formulas on trees Source-Consistency: Given a source structure I in K S, is there a target J in K T s.t. (I,J) in τ ? Typechecking: Decide if for all I in K S and all J s.t. (I,J) in τ, J is in K T. Composition of settings ? Query Answering: Given a source structure I in K S, decide if for all J s.t. (I,J) in τ, J is in K Q. Classical Data-Exchange

6. 6 :c Deterministic Transducer on unranked trees with attributes. In practice, XSLT program. Generalization to non-deterministic Transducers.. Class τ defined by Transducers 00011110 0*1* cabababcaaaaa. c(ab)*ca* 0:ab abababaaaaab c(ab)*ca* 1:a 0:ab 1:a 0:c ababaaa + abcaaa + cabaaa + ccaaa c(ab)*ca* 00111 0*1* 0:ab 1:a c* ab c* a c* a c* 011

7. 7 (K S,τ,K T ) is a setting, where τ is a transducer: ε-Source-Consistency: Given a source structure I, is there a source I’  K S, ε-close to I s.t. τ(I’) is ε-close to K T ? ε-Typechecking: Decide if for all I in K S, τ(I) is ε-close to K T. ε-Composition of settings. General transducer τ : ε-Query Answering: Given a source structure I, is there a source I’ ε-close to I s.t. any J [s.t. (I’,J) is in τ] is ε- close to K Q ?. Approximate Data Exchange

8. 8 Let F be a property on a class K of structures U An ε -tester for F is a probabilistic algorithm A such that: If U |= F, A accepts If U is ε-far from F, A rejects with high probability A property F is testable if there exists a probabilistic algorithm A s.t. For all ε it is an ε -tester for F Time(A) independent of n=|U|. R. Rubinfeld, M. Sudan, Robust characterizations of polynomials, 1994 O. Goldreich, S. Goldwasser and D. Ron, Property Testing and its connection to Learning and Approximation, 1996.Property Testing and its connection to Learning and Approximation Tester usually implies a linear time corrector. (ε 1, ε 2 )- Tolerant Tester. 2. Property Testing

9. 9 1.Satisfiability: T |= F 2.Approximate Satisfiability: T |= F 3.Approximate Equivalence: Image on a class K of trees Approximate Satisfiability and Equivalence

10. 10 1.Classical Edit Distance: Insertions, Deletions, Modifications 2.Edit Distance with moves. 0111000011110011001 0111011110000011001 3. Edit Distance with Moves generalizes to Ordered Trees Edit Distances with Moves

11. 11 Uniform Statistics: k=1/ε Distance between words ( NP-complete) Testable, O(1): Sample N subwords of length k: Y(W) and Y(W’) If |Y(w)-Y(w’)| 1 < ε accept, else reject W=001010101110 length n, n-k+1 blocks of length k For k=2, n=12, 11 blocks Fact 1: dist(W,W’)  |u.stat(W)-u.stat(W’)| 1 for words of similar length Fact 2: |u.stat(W)-Y(W) | 1 ≤  for Y(W) the u.stat vector on N samples

12. 12 r = (010)*0*1* + 1*(01)*(110)* Statistics on Regular Expressions Y(w) H={u.stat(w) : w in r } is a union of polytopes. 2 polytopes for r.. Membership Tester: Compute Y(w). Accept if d(Y(w),H) ≤ , else reject k=2

13. 13 ε-Source-Consistency : Given a source structure I, is there a source I’  K S ε- close to I s.t. τ(I’) is ε-close to K T ? Complexity parameter: n=|I| Case of 1-state on words: how to k-sample uniformly in τ(I) ? Suppose τ(0)=a, τ(1)=bbb. Adjust the probabilities: If s=0…, 1 possible block from τ(0), adjust with 1/3 If s=1…, 3 possible blocks from τ(1), choose a shift in {0,1,2} uniformly Approximate u.stat( τ(I)). 3. Approximate Data Exchange I = 0 0 0 0 1 1. τ(I) = a a a a b b b b b b

14. 14 Analysis of  for ε-Source-consistency: u.stat(I)  1 (u 1 )+ 2 (u 2 )+ 3 (u 3 ) u.stat(   (I))=  (v 1 )+  ’(v 4 )+ 2 (v 2 )+ 3 (v 3 ) with  +  ’= 1. (u1)(u1) (u2)(u2) (u3)(u3) (I)(I) HH HSHS H S  u.stat(K S ) H   u.stat(   ) H T  u.stat(K T ) u 1 :v 1 q1q1 u 2 :v 2 q2q2 u 3 :v 3 q3q3 u 1 :v 4 q4q4  11 22

15. 15 Tester for ε-Source-consistency: 1-1-   =0,  ’= 1  = 1,  ’=0 HTHT Tester: u.stat(I) is ε-far from H S : reject [I is far from K S ]  Tester for K S. Generate  ={  | u.stat(I) is ε-close from being decomposable over H  }  Testers for K  While (  ≠  ) { take a  in , approximate u.stat(   (I)) and x=d(u.stat(   (I)), H T ) If x≤ , then accept and stop else remove  from  } Reject Find I’: If the test accepts, split 1 with the  proportions : I = u 2 u 1 u 1 u 1 u 1 u 1 u 1 u 1 u 1 u 1 u 3 u 3 u.stat(   (I))=  (v 1 )+  ’(v 4 )+ 2 (v 2 )+ 3 (v 3 ) with  +  ’= 1. I’ = u 1 u 1 u 1 u 2 u 3 u 3 u 1 u 1 u 1 u 1 u 1 u 1

16. 16 Lemma: If I is s.t.  (I)  K T, then A accepts because there is a  with dist(   (I),K T )=0 Lemma: If I is ε-far from being Source-Consistent, then the tester reject with high probabilities. Theorem: For every ε > 0, there is an ε-tester for the ε-Source-Consistency on words. Corollary: If I is ε-Source-Consistent, the procedure leads to an I’ s.t.  (I’) is  -close to K T. Approximate ε-Source-Consistency:

17. 17 Given a source structure I, is there a source I’ ε-close to I s.t. τ(I’) is ε-close to K T ? Case of 1-state: how to k-sample uniformly in τ(I) ? Suppose τ(0)=ab, τ(1)=a, τ(2)=ccc. Adjust the probabilities: If s=0…,2 possible blocks from τ(0), adjust with 1/3 If s=1…, 1 possible block from τ(1), adjust with 1/6 If s=2…, 3 possible blocks from τ(2), adjust with 1/2 Approximate ustat( τ(I)). ε-Source-Consistency 0 0 2 1 1 1 1 a b a b c c c a a a a aa : 4/7.1/6.3/4=1/14 ab : 2/7.1/3.1/2=1/21 Outputs: bc : 1/7.1/3.1/2=1/42  ustat( τ(w))= ca : 1/7.1/3.1/2=1/42 cc : 1/7.1/2.2/3=1/21

18. 18 Image of the statistics by a general transducer τ I τ (I) Union of polytopes Applications: ε-Source-Consistency: ε-Query Answering: d( u.stat[τ(I)],H T ) ≤  ?u.stat[τ(I)]  ε H Q ? u.stat(I)=

19. 19 Inclusion Tester for regular properties Time polynomial in m=Max(|r 1 |,|r 2 |): Application : ε-Typechecking: Decide if J is ε-close to K T [for all I in K S and all (I,J) in τ]. Solution: Inclusion Tester for τ(K S )   K T.

20. 20 Statistics on Trees (1(1,1),.) (1,.) T: Ordered (extended) Tree of rank 2. T’: squeleton W: word with labels. Apply u.stat on W and define u.stat(T).

21. 21 Extension to trees Statistics on DTDs: H={stat(t) : t in DTD} is still a union of polytopes (harder analysis to construct it) Transducer  with attributes:  :  S ×Q  Hedge  T,A T [Q] h :  S ×Q×A S  {1}  Var extended to  S ×Q×Str  Str  Var  :  S ×Q×A T ×D T  {1,…,k} where D T is the hedge defined by .  is decomposable in a finite number of paths in the graph of the strongly connected components. Lemma: The image of a statistical vector through a path is a union of polytopes.

22. 22 ε-Source-Consistency on trees Test: If there is a  (allowing a decomposition of t on H  ) s.t. u.stat(   (t)) is  -close to H T then accept, else reject Lemma: If  (t)  K T, then there is a  with dist(   (t),K T )=0. Lemma: If t is ε-far from being ε-Source-Consistent, then we reject with high probabilities.  Testers for K S, K  ;  x:approximation of u.stat(   (t)), d(x,H T ) ≤  ? Theorem: For every ε > 0, there is an ε-tester for the ε-Source-Consistency on trees. Corollary: If t is ε-Source-Consistent, the procedure leads to an t’ s.t.  (t’) is  -close to K T

23. 23 Composition of close settings An ε-corrector for a class K 0  K is a algorithm A which takes as input a structure I which is ε-close to K 0 and outputs a structure I 0  K 0, such that I 0 is ε-close to I. Ex : If an XML file F is ε-close from a DTD, find a valid F’ ε-close to F: http://www.lri.fr/~mdr/xml/ http://www.lri.fr/~mdr/xml/ Data Exchange settings: (K S1,τ 1,K T1 ), (K S2,τ 2,K T2 ): Solution if they are ε-composable –K T1 and K S2 are ε-close. –the settings satisfy ε-typechecking Composition: Apply correctors at every stage to define the new τ.  (K S1,τ,K T2 ) satisfies 3ε-typechecking.

24. 24 τ2τ2 Composition τ1τ1 C1C1 C C2C2 τ = C 2 ◦ τ 2 ◦ C ◦ C 1 ◦ τ 1 K T1 K S2 K T2

25. 25 Conclusion 1.Data Exchange: –Source-Consistency, –Typechecking, –Query-Answering. 2.Approximate Data Exchange: Property Testing based Approximation –ε-Source-Consistency, –ε-Typechecking, –ε-Query-Answering, –ε-Composition.

26. 26 Questions ? Adrien Vieilleribière: vieille@lri.frvieille@lri.fr Michel de Rougemont: mdr@lri.frmdr@lri.fr