1. Local Parallel Rewriting : Theory and Applications * Giorgio Satta University of Padua * Joint work with : D. Melamed, O. Rambow, B. Wellington

2. FG04Nancy2 Part I : Introduction –Parallel rewriting –Locality Part II : Theory –Descriptional complexity –Normal forms and language hierarchy Part III : Applications –Synchronous rewriting –Translation Algorithms Introduction Summary :

3. FG04Nancy3 In Computational Linguistics, formal grammars are used to model the syntactic structure of natural language sentences Syntactic modeling is fundamental to applications such as natural language understanding, machine translation, etc. We can abstractly view a grammar as a set of elementary objects (productions) that represent descriptions of basic syntactic relations. These objects are then combined in order to obtain syntactic structures Motivations

4. FG04Nancy4 Example : Context-Free Grammars (CFGs) S[went]  NP[pat] VP[went] VP[went]  VP[went] Adv[early] VP[went]  V[went] NP[home] V[went]  went Motivations S[went] Adv[early] VP[went]NP[pat] VP[went] NP[home] V[went] went

5. FG04Nancy5 So called “long distance” relations cannot be directly represented by CFGs Example : extraction what did Alice eat ? S[eat]  NP[what] S[eat] VP[eat]  V[eat] NP[e] There is no direct dependence between the two productions above Motivations

6. FG04Nancy6 Several other constructions are problematic for CFGs Topicalization I enjoyed the soup, but the main course I thought was awful Clitic climbing Mari lo queria terminar de hacer Scrambling … dass den Kuhlschrank niemand zu reparieren versprochen hat Motivations

7. FG04Nancy7 A related problem arises in machine translation, where we want to relate phrases that correspond under the translation Example : damoy Pat rano pashol Pat went home early Motivations

8. FG04Nancy8 Several solutions have been proposed in the literature : Enriching CFGs with feature structures (HPSG, LFG, …) Use of logical constraints (ID/LP, linearization grammars) Use of special purpose operators (domain union, HPSG) Local parallel rewriting (TAG, LCFRS, …) Motivations

9. FG04Nancy9 A parallel rewriting system defines elementary objects that allow simultaneous rewriting of a sentential form at several places : Example : TAG Parallel rewriting Adjunction

10. FG04Nancy10 The simplest way we can implement parallel rewriting is by using tuples of context-free productions (A 1   1, …, A r   r ) and define rewriting as the simultaneous application of all the production components  1 A 1  2 …  r -1 A r  r    1  1  2 …  r -1  r  r Parallel rewriting

11. FG04Nancy11 Several parallel rewriting systems have been defined in the literature, as for instance Matrix Grammars, Vector Grammars, Scattered Context Grammars, etc. These systems are too powerful. They can generate NP-complete languages Non-semilinear languages Parallel rewriting

12. FG04Nancy12 Local rewriting is a restriction that requires each application of a production to rewrite symbols that have been introduced in a single step in a sentential form Example : (A  ABA, A  BA ) non-local Local parallel rewriting  A B A C B A  A B A B A C B B A A C A

13. FG04Nancy13 Local rewriting can be implemented using superscript indices to distinguish among different occurrences of the same symbol Example : (A  A (1) B (1) A (1), A  B (1) A (1) ) A (1) C (1) A (1)  A (2) B (2) A (2) C (1) B (2) A (2)  A (3) B (3) A (3) B (2) A (2) C (1) B (2) B (3) A (3) Local parallel rewriting

14. FG04Nancy14 By combining parallel rewriting and local rewriting we obtain a local parallel grammar Example : damoy Pat rano pashol lit: home Pat early went Pat went home early (S[pashol]  VP[pashol] (1) NP[pat] (2) VP[pashol] (1) ) (VP[pashol]  VP[pashol] (1), VP[pashol]  Adv[rano] (2) VP[pashol] (1) ) (VP[pashol]  NP[damoy] (1), VP[pashol]  V[pashol] (2) ) Local parallel rewriting

15. FG04Nancy15 Example (cont’d) : S[pashol] (1)  VP[pashol] (2) NP[pat] (3) VP[pashol] (2)  VP[pashol] (4) NP[pat] (3) Adv[rano] (5) VP[pashol] (4)  NP[damoy] (6) NP[pat] (3) Adv[rano] (5) V[pashol] (7)  * damoy Pat rano pashol Local parallel rewriting

16. FG04Nancy16 Parallelism and locality have been exploited in many rewriting systems that have been independently defined in the literature, motivated by different application domains All these superficially different systems were later shown to be generatively equivalent This provides evidence that parallel rewriting and local rewriting are “natural” notions Local parallel rewriting

17. FG04Nancy17 Known local parallel rewriting systems : Syntax-directed compilers –Deterministic Tree-Walking Transducers (Aho and Ullman, 1971) Visual and relational languages, syntactic pattern matching and biological data modeling –String generating Context-Free Hypergraph Grammars (Bauderon & Courcelle, 1987; Habel & Kreowsky, 1987) Formal language and translation theory –Finite Copying Top-Down Tree-to-String Transducers (Engelfriet et al., 1980) –Local Unordered Scattered Grammars (Rambow & Satta, 1998) Local parallel rewriting

18. FG04Nancy18 Known local parallel rewriting systems (cont’d) : Natural language processing –Linear Context-Free Rewriting Systems (Vijay-Shanker, Weir & Joshi, 1987) –Multiple Context-Free Grammars (Kasami et al., 1987) –Multi-Component Tree Adjoining Grammars (Weir, 1988) –Finite-copying Lexical Functional Grammars (Seki et al., 1993) –Minimalist Grammars (Stabler, 1997) –Simple Range Concatenation Grammars (Boullier, 1998) Local parallel rewriting

19. FG04Nancy19 Introduction Summary : Part I : Introduction –Parallel rewriting –Locality Part II : Theory –Descriptional complexity –Normal forms and language hierarchy Part III : Applications –Synchronous rewriting –Translation Algorithms

20. FG04Nancy20 Languages generated by local parallel rewriting systems have the following important properties : Are included in the class of Context-Sensitive Languages Can be parsed in deterministic polynomial time Are semilinear These languages belong to the class of Mildly Context Sensitive Languages (Joshi 1985; Joshi, Vijay-Shanker & Weir, 1991) Theory

21. FG04Nancy21 In a local parallel rewriting system, derivations can be described by the trees generated by a context-free grammar : Theory ( S[pashol]  VP[pashol] (1) N[pat] (2) VP[pashol] (1) ) ( N[pat]  Pat )... ( VP[pashol]  VP[pashol] (1), VP[pashol]  Adv[rano] (2) VP[pashol] ) ( VP[pashol]  NP[damoy] (1), VP[pashol]  V[pashol] (2) )

22. FG04Nancy22 Parallelism and locality can be viewed as resources and can therefore be measured Degree of parallelism = fan-out –max number of components in productions Degree of locality = rank –max number of productions that can rewrite a local domain –max branching in underlying derivation Descriptional Complexity

23. FG04Nancy23 Example : Descriptional Complexity rank fan-out

24. FG04Nancy24 Examples : Linear Context-Free Languages : f = 1, r = 1 Context-Free Languages : f = 1, r = 2 Tree-Adjoining Languages : f = 2, r = 2 Questions: Are there normal forms with bounded fan-out ? Are there normal forms with bounded rank ? Can we “trade” the two resources ? Descriptional Complexity

25. FG04Nancy25 Fan-out = f, rank = r Language Hierarchy LCFL nl-CFL TAL CCL3 f = 1 2 3 4 r = 1 2... 3 4 ET0L f2

26. FG04Nancy26 Introduction Summary : Part I : Introduction –Parallel rewriting –Locality Part II : Theory –Descriptional complexity –Normal forms and language hierarchy Part III : Applications –Synchronous rewriting –Translation Algorithms

27. FG04Nancy27 A synchronous grammar is a formal model of translation The model combines grammars by pairing productions that represent corresponding phrases Rewriting is carried out synchronously Examples : Finite Transducers Syntax Directed Translation Schemata Synchronous Tree Adjoining Grammars Synchronous rewriting

28. FG04Nancy28 Synchronous rewriting Example : ongaku wo kiku lit: music to listening listening to music [ VP[listening]  V[listening] (1) PP[music] (2), VP[kiku]  PP[ongaku] (2) V[kiku] (1) ] [ PP[music]  P[to] (1) NP[music] (2), PP[ongaku]  NP[ongaku] (2) P[wo] (1) ]

29. FG04Nancy29 Synchronous grammars can be viewed as specific applications of local parallel rewriting : Parallelism is exploited to rewrite on several dimensions Locality is exploited to enforce synchronous rewriting In this way we have a common and well-understood framework for investigating synchronous rewriting and for comparing already known synchronous formalisms Synchronous rewriting

30. FG04Nancy30 Synchronous rewriting Example : damoy Pat rano pashol lit: home Pat early went Pat went home early [ (S[pashol]  VP[pashol] (1) NP[pat] (2) VP[pashol] (1) ), (S[went]  NP[pat] (2) VP[went] (1) ) ]... [ (VP[pashol]  NP[damoy] (1), VP[pashol]  V[pashol] (2) ), (VP[went]  V[went] (2) NP[home] (1) ) ]

31. FG04Nancy31 Synchronous rewriting Example (cont’d) : [ S[pashol] (1), S[went] (1) ]  [ VP[pashol] (2) NP[pat] (3) VP[pashol] (2), NP[pat] (3) VP[went] (2) ]  [ VP[pashol] (4) NP[pat] (3) Adv[rano] (5) VP[pashol] (4), NP[pat] (3) VP[went] (4) Adv[early] (5) ]  [ NP[damoy] (6) NP[pat] (3) Adv[rano] (5) V[pashol] (7), NP[pat] (3) V[went] (7) NP[home] (6) Adv[early] (5) ]  * [ damoy Pat rano pashol, Pat went home early ]

32. FG04Nancy32 Translation problem for synchronous grammar G = ‹G 1, G 2 › : Input string u Output set T (G, u ) of all strings that translate u (and their derivations) We encode T (G, u ) as a local parallel grammar Translation

33. FG04Nancy33 Assume synchronous grammar G = ‹G 1, G 2 › : L (G j ) = language freely generated by grammar component G j T j = projection on the j – th dimension of translation T (G ) In general we have L (G j ) ≠ T j We can construct a local parallel grammar auto-proj(G, j ) that generates T j Weak language preservation property Translation

34. FG04Nancy34 We can intersect a synchronous translation T (G ) with relation L (M 1 ) × L (M 2 ), where M 1, M 2 are finite automata This is done through a generalization of a construction for CFG due to (Bar-Hillel et al., 1964) The result is a synchronous grammar Translation

35. FG04Nancy35 Algorithm : Construct G  by intersecting G and finite automata M 1 and M 2, where M 1 generates {u} and M 2 generates  * Construct local parallel grammar auto-proj(G , 2) Translation

36. FG04Nancy36 Conclusions Local parallel rewriting systems have been used in several fields of Computer Science These languages form a two-dimensional non-collapsing hierarchy on the fan-out and rank parameters Synchronous rewriting can be viewed as a specific application of local parallel rewriting Translation algorithms can be developed on the basis of already known parsing techniques