1. Martin KayCL Introduction1 Martin Kay Stanford University Ling 138/238

2. Martin KayCL Introduction2 30Introduction Oct1Complexity; String search 6Knuth-Morris-Pratt; Boyer Moore; 8Suffix Trees 13Tagging; Alignment 15 20Chomsky Hierarchy; Regular Expressions 22 27Finite-state automata 39

3. Martin KayCL Introduction3 Nov3Morphology 5 10Context-free grammar 12 17Unification, HPSG, LFG 19 24Machine Translation 26 Dec1Summary; Wrap-up 3

4. Martin KayCL Introduction4 Martin Kay KAY@csli.stanford.edu 740 3043 Margaret Jacks 124 Office hours: TuTh 4.15-5.45 p.m. Linguistics 138/238

5. Martin KayCL Introduction5 Prerequisites and Expectations No prerequisites Classroom participation Occasional readings Learn Prolog Laboratory sessions Homework Problems Project

6. Martin KayCL Introduction6 Project Learn something new about language Significant programming Group work Modifying or amplifying existing code A HMM-based tagger A searcher for tagged text Implementation of Suffix trees Morphological analysis Named-entity recognition

7. Martin KayCL Introduction7 Intellectual Relations Relation to —Linguistics —Psychology —Artificial Intelligence —Computer Science

8. Martin KayCL Introduction8 Computational Linguistics as Science

9. Martin KayCL Introduction9 Ideas from Computing Search Divide and Conquer Guides and Oracles Nondeterminism Dynamic Programming Scheduling, agendas Compilation Unification Automata Theory Co-routining and parallelism Top-down vs. bottom-up Complexity

10. Martin KayCL Introduction10 Ideas from Computing Search Nondeterminism Dynamic Programming

11. Martin KayCL Introduction11 A Maize Keep you right hand on the wall Search Nondeterminism Dynamic Programming

12. Martin KayCL Introduction12 Backup! A Maize Backup! Out! Search Nondeterminism Dynamic Programming

13. Martin KayCL Introduction13 Nondeterminism A process is nondeterministic if there are points in it when a choice must be made, but the information necessary to make the choice is not available. Solution: Pick one of the alternatives. If it does not work out, come back and pick another one. Note: the information required to make the choice was available after all! Search Nondeterminism Dynamic Programming

14. Martin KayCL Introduction14 Dynamic Programming pour f 1234 o2123 r3222 Paris Dijon Mulhouse Strasbourg ChalonsMetz 266 192 161 344 276 115 234 288 458 620 619 Search Nondeterminism Dynamic Programming

15. Martin KayCL Introduction15 peoplenpnpnp ss s likepreppppp vvpvp thedetnpnp Frenchadjnn drinkn vp The CKY Chart Context free: All phrase with the same — Coverage, and — Category enter into larger phrases as a single item Context free: All phrase with the same — Coverage, and — Category enter into larger phrases as a single item Search Nondeterminism Dynamic Programming

16. Martin KayCL Introduction16 Ideas from Computing Unification

17. Martin KayCL Introduction17 Unification AttributeReport 1Report 2Combined Report eyesblueblueblue hairblack or brownbrown or redbrown accentItalianItalian wifesee belowsee belowsee below childrenAhemed & AngelaRebecca & AngelaAhmed, Angela & Rebecca agemiddle48Middle Wife eyesbrownbrown weight247 lbs112 Kg247 lbs dispositionsurlysurly Unification

18. Martin KayCL Introduction18 Unification AttributeReport 1Report 2Combined Report eyesblueblueblue hairblack or brownbrown or redbrown accentItalianItalian wifesee belowsee belowsee below childrenAhemed & AngelaRebecca & AngelaAhmed, Angela & Rebecca agemiddle48Middle Wife eyesbrowngreyFAIL weight247 lbs112 Kg247 lbs dispositionsurlysurly Unification

19. Martin KayCL Introduction19 English Agreement The dog sleeps The dogs sleep The dog slept The dogs slept The sheep sleeps The sheep sleep The sheep slept The sheep that was in the barn slept The sheep that were in the barn slept Unification

20. Martin KayCL Introduction20 German Case Der Junge sah den Lehrer Den Lehrer sah der Junge Das Mädchen sah der Junge der Junge sah das Mädchen Die Lehrerin sah den Lehrer Die Lehrerin sah das Mädchen Unification

21. Martin KayCL Introduction21 Ideas from Computing Finite-State Methods

22. Martin KayCL Introduction22 Finite-State Methods in Language Processing The Application of a branch of mathematics —The regular branch of automata theory to a branch of computational linguistics in which what is crucial is (or can be reduced to) —Properties of string sets and string relations with —A notion of bounded dependency Finite-State Methods

23. Martin KayCL Introduction23 Applications Finite Languges —Dictionaries —Compression Phenomena involving bounded dependency —Morpholgy Spelling Hyphenation Tokenization Morphological Analysis —Phonology Approximations to phenomena involving mostly bounded dependency —Syntax Phenomena that can be translated into the realm of strings with bounded dependency —Syntax Finite-State Methods

24. Martin KayCL Introduction24 Ideas from Computing Complexity

25. Martin KayCL Introduction25 The Chomsky Hierarchy GrammarLanguageAutomaton Type 0Recursively Turing Machines Enumerable Sets Context-sensitiveContext-sensitiveNondeterministic linear space bound Turing Machines Context-freeContext-freeNondeterministic push- down automata LR(k)Deterministic Context-Deterministic push-down free automata Regular ExpressionsRegular SetsFinite-state automata Left (Right) Linear Complexity

26. Martin KayCL Introduction26 Computation and Psychology Sentence Processing

27. Martin KayCL Introduction27 Computational Linguistics as Engineering

28. Martin KayCL Introduction28 Tools for Linguists TLF, OED Corpus Linguistics Field Notes Grammar Testing

29. Martin KayCL Introduction29 Translation MT, Translator's Tools Alignment, Dictionaries, Term Banks Normalization and Tuning

30. Martin KayCL Introduction30 Other Applications Writer's Tools —Spelling —Dictionary, Thesaurus —Grammar Natural Language Interfaces Information Storage and Retrieval

31. Martin KayCL Introduction31 CL & AI Text Interpretation Meaning Linguistics ??? Text, Meaning, and Interpretation