1. NLP Syntax1 Syntax The Structure of language Dave Inman

2. NLP Syntax2 1. Outline Why is the structure of language (syntax) important? What knowledge is required to understand natural language? How do we represent syntax? What does an example grammar for English look like? What strategies exist to find the structure in natural language? A Prolog program to recognise English sentences

3. NLP Syntax3 2. Why is the structure of language (syntax) important? Consider an alternative (ELIZA) that uses no structure http://www-ai.ijs.si/eliza-cgi-bin/eliza_script See if you can work out 1. What it does well 2. What is does badly 3. What kinds of patterns it looks for in your input

4. NLP Syntax4 2.1. Pattern matching as an alternative (eg. Eliza) This uses a database of input output pairs. The input part of pair is a template to be matched against the user input The output part of the pair is given as a response. Very flexible Accepts non grammatical utterances for example. X computers Y => Do computers interest you? X mother Y => Tell me more about your family? This is easy to program A couple of pages of Prolog code for example.

5. NLP Syntax5 2.1. Pattern matching as an alternative (eg. Eliza) The input part of pair is a template to be matched against the user input The output part of the pair is given as a response. But… Nothing is known about structure (syntax) I X you => Why do you X me? Fine for X = like, but not for X = do not know Nothing is known about meaning (semantics) I feel X => I'm sorry you feel X. Fine for X = depressed, but not for X = happy Conclusion : We need both syntax & semantics to understand natural language.

6. NLP Syntax6 2.2. Syntax shows the role of words in a sentence. John hit Sue vs Sue hit John Here knowing the subject allows us to know what is going on.

7. NLP Syntax7 2.3. Syntax shows how words are related in a sentence. Visiting aunts ARE boring. vs Visiting aunts IS boring. Subject verb agreement allows us to disambiguate here.

8. NLP Syntax8 2.4. Syntax shows how words are related between sentences. (a) Italy was beating England. Germany too. (b) Italy was being beaten by England. Germany too. Here ellipsis (missing parts of a sentence) does not allow us to understand the second sentence. But syntax allows us to see what is missing.

9. NLP Syntax9 2.5. But syntax alone is not enough Visiting museums can be boring This is not ambiguous for us, as we know there is no such thing as a "visiting museum", but syntax cannot show this to a computer. Compare with… Visiting aunts can be boring

10. NLP Syntax10 3. What knowledge is required to understand natural language? Syntax What is the difference between… Flying planes ARE dangerous Flying planes IS dangerous Semantics Meaning of words / phrases/ sentences/ whole texts. The sentence “She ran to the bank” is ambiguous. Compare: She owed him £50 so.....she ran to the bank. Her son fell over by the river so...she ran to the bank.

11. NLP Syntax11 3. What knowledge is required to understand natural language? Pragmatics How language is used. The sentence: Can you open the door… could be: A polite request A question Which is it more likely to be? Why?

12. NLP Syntax12 3. Errors to show knowledge needed for NLP Signs in a restaurant The manager has personally passed all the water served here. Our wines leave you nothing to hope for. Closing down thanks to all our customers. We highly recommend the restaurant tart.

13. NLP Syntax13 3. Errors to show knowledge needed for NLP Signs in a hotel You are invited to take advantage of the chambermaid. Please to bathe inside the tub. Please leave your values at the front desk. If this is your first visit to the U.S.S.R. you are welcome to it. Customers should note that any complaints about rudeness in the staff will be dealt with very severely.

14. NLP Syntax14 3. Errors to show knowledge needed for NLP Signs in a dry cleaners Drop your trousers here for best results. If we have failed you in any way we shall be happy to do it again. Gent’s trousers slashed. Ladies: leave your clothes here and spend the afternoon having a good time.

15. NLP Syntax15 3. Errors to show knowledge needed for NLP Various Signs We take your bags and send them in all directions (airline). Correctly English in 100 days (book). Don’t handle the fruit. Ask for Debbie (UK greengrocer). Why not rent out a movie for a dull evening (video shop).

16. NLP Syntax16 4. How do we represent syntax? 4.1. Parse Tree

17. NLP Syntax17 4. How do we represent syntax? 4.2. List Sue hit John [ s, [np, [proper_noun, Sue] ], [vp, [v, hit], [np, [proper_noun, John] ]

18. NLP Syntax18 5. What does an example grammar for English look like? 5.1. Re-write rules sentence -> noun phrase, verb phrase noun phrase -> noun noun phrase -> determiner, noun verb phrase -> verb, noun phrase

19. NLP Syntax19 5. What does an example grammar for English look like? 5.2. Transition networks Not so popular now, but used in the past with ATN parsers for example. They show how to traverse a sentence with allowable structures:

20. NLP Syntax20 6. An exercise: Parse the sentence: "They are cooking apples." Give two parses.

21. NLP Syntax21 6. Parse 1

22. NLP Syntax22 6. Parse 2

23. NLP Syntax23 7. What strategies exist for trying to find the structure in natural language? 7.1. Top Down vs. Bottom Up Bottom - Up John, hit, the, cat prpn, hit, the, cat prpn, v, the, cat prpn, v, det, cat prpn, v, det, n np, v, det, n np, v, np np, vp s Better if many alternative rules for a phrase Worse if many alternative terminal symbols for each word Top - Down s s -> np, vp s -> prpn, vp s -> John, v, np s -> John, hit, np s -> John, hit, det,n s -> John, hit, the,n s -> John, hit, the,cat Better if many alternative terminal symbols for each word Worse if many alternative rules for a phrase

24. NLP Syntax24 7. What strategies exist for trying to find the structure in natural language? 7.2. Depth First vs. Breadth First Depth First Try rules one at a time and back track if you get stuck Easier to program Less memory required Good if parse tree is deep Breadth First Try all rules at the same time Can be faster Order of rules is not important Good if tree is flat

25. NLP Syntax25 7. What strategies exist for trying to find the structure in natural language? 7.3. Left - Right vs. Right – Left Left - Right Take words from left to right Take rule constituents from left to right Right - Left Take words from right to left Take rule constituents from right to left Left - Right usually best for a language like English where subject comes before verb ; good for subject - verb agreement; speech & real time input is L->R; closer to human processing. We have trouble with "Have the students given their assignments by their lecturers" for this reason.

26. NLP Syntax26 7. What strategies exist for trying to find the structure in natural language? 7.4. A simple Prolog parser's strategy Top Down Prolog tries to satisfy the query "Is this a sentence?" and works top down in a search for resolution of this query. Depth first Prolog's in built search strategy is depth first, so a simple parser uses this. L->R The grammar rules are taken from left to right, and so are the words of phrases / sentences as Prolog tries to match them against the grammar rules.

27. NLP Syntax27 8. What does a Prolog program look like that tries to recognise English sentences? s --> np vp. np --> det n. np --> det adj n. vp --> v np.

28. NLP Syntax28 8. What does a Prolog program look like that tries to recognise English sentences? sentence(S) :- noun_phrase(NP), verb_phrase(VP), append(NP,VP,S). noun_phrase(NP) :- determiner(D),noun(N),append(D,N,NP). noun_phrase(NP) :- determiner(D),adj(A),noun(N),append(D,A,AP),appen d(AP,N,NP). verb_phrase(VP) :- verb(V), noun_phrase(NP), append(V,NP,VP). determiner([D]) :- member(D,[the,a,an]). noun([N]) :- member(N,[cat,dog,mat,meat,fish]). adj([A]) :- member(A,[big,fat,red]). verb([V]) :- member(V,[ate,saw,killed,pushed]).