LING 388: Computers and Language Lecture 30

Last class today Reminder: TCE survey! submit your term project (Friday); have already received some!

nltk book: chapter 5 (Adjectives and Adverbs) (Attributive) The falling man … / (Predicative) The man is falling … Difference in meaning sometimes: Mary's really nice teacher Mary's teacher is really nice

nltk book: chapter 5 (Exploring Tagged Corpora) Words that follow 'often', using .words(): Words categories that follow 'often', using .tagged_words():

nltk book: chapter 5 (Trigram search) from nltk.corpus import brown brown.tagged_sents() [[('The', 'AT'), ('Fulton', 'NP-TL'), ('County', 'NN-TL'), ('Grand', 'JJ- TL'), ('Jury', 'NN-TL'), ('said', 'VBD'), ('Friday', 'NR'), ('an', 'AT'), ('investigation', 'NN'), ('of', 'IN'), ("Atlanta's", 'NP$'), ('recent', 'JJ'), ('primary', 'NN'), ('election', 'NN'), ('produced', 'VBD'), ('``', '``'), ('no', 'AT'), ('evidence', 'NN'), ("''", "''"), ('that', 'CS'), ('any', 'DTI'), ('irregularities', 'NNS'), ('took', 'VBD'), ('place', 'NN'), ('.', '.')], [('The', 'AT'), ('jury', 'NN'), ('further', 'RBR'), ('said', 'VBD'), ('in', 'IN'), ('term-end', 'NN'), ('presentments', 'NNS'), ('that', 'CS'), ('the', 'AT'), ('City', 'NN-TL'), ('Executive', 'JJ-TL'), ('Committee', 'NN-TL'), (',', ','), ('which', 'WDT'), ('had', 'HVD'), ('over-all', 'JJ'), ('charge', 'NN'), ('of', 'IN'), ('the', 'AT'), ('election', 'NN'), (',', ','), ('``', '``'), ('deserves', 'VBZ'), ('the', 'AT'), ('praise', 'NN'), ('and', 'CC'), ('thanks', 'NNS'), ('of', 'IN'), ('the', 'AT'), ('City', 'NN-TL'), ('of', 'IN-TL'), ('Atlanta', 'NP-TL'), ("''", "''"), ('for', 'IN'), ('the', 'AT'), ('manner', 'NN'), ('in', 'IN'), ('which', 'WDT'), ('the', 'AT'), ('election', 'NN'), ('was', 'BEDZ'), ('conducted', 'VBN'), ('.', '.')], ...]

nltk book: chapter 5 (Trigram search) >>> for ts in brown.tagged_sents(): ...     for (w1,t1), (w2,t2), (w3,t3) in nltk.trigrams(ts): ...             if t1.startswith('V') and t3.startswith('V') and t2 == 'TO': ...                     print(w1,w2,w3) ...  combined to achieve continue to place serve to protect … … help to intensify seems to overtake want to buy 4023 such fragments

nltk book: chapter 5 (tag ambiguity > 3)

nltk book: chapter 5 (tag ambiguity > 3) import nltk from nltk.corpus import brown bnt = brown.tagged_words(categories='news', tagset='universal') cfd = nltk.ConditionalFreqDist((w.lower(),tag) for (w,tag) in bnt) cfd <ConditionalFreqDist with 13112 conditions> cfd['bank'] FreqDist({'NOUN': 16}) cfd['grant'] FreqDist({'NOUN': 11, 'VERB': 4}) len(cfd['grant']) 2

nltk book: chapter 5 (tag ambiguity > 3) >>> for w in sorted(cfd.conditions()): ... if len(cfd[w]) > 3: ... print(w,cfd[w]) ... best <FreqDist with 4 samples and 31 outcomes> close <FreqDist with 4 samples and 12 outcomes> open <FreqDist with 4 samples and 33 outcomes> present <FreqDist with 4 samples and 30 outcomes> that <FreqDist with 4 samples and 829 outcomes>

nltk book: chapter 5 (tag ambiguity > 3) >>> for w in sorted(cfd.conditions()): ... if len(cfd[w]) > 3: ... print(w,cfd[w].most_common()) ... best [('ADJ', 28), ('ADV', 1), ('NOUN', 1), ('VERB', 1)] close [('ADV', 6), ('ADJ', 3), ('VERB', 2), ('NOUN', 1)] open [('ADJ', 13), ('VERB', 11), ('NOUN', 8), ('ADV', 1)] present [('ADJ', 21), ('ADV', 7), ('NOUN', 1), ('VERB', 1)] that [('ADP', 546), ('DET', 150), ('PRON', 128), ('ADV', 5)]