Are Linguists Dinosaurs? 1.Statistical language processors seem to be doing away with the need for linguists. –Why do we need linguists when a machine can learn the rules of the language and parse sentences? 2.However, (workable) statistical parsers require a parsed corpus (a treebank). –E.g. Charniak and Collins parsers use Penn Treebank. 3.Treebanks are produced (primarilly) by linguists: –texts parsed with symbolic parsers (e.g. Fidditch) with grammar rules and lexicons constructed by linguists –Results corrected by hand (again by linguistically trained people). 4.Point: statistical parsers could not exist without the input of linguists.

Are Linguists Dinosaurs? Also, many of the most sucessful parsing (and MT) systems are hybrid systems: –human-written grammar/lexicon –corpus-derived stats for disambiguation. For languages other than English, there is still much work for linguists developing the needed symbolic grammars and lexicons needed by statistical systems.

Are Linguists Dinosaurs? Once a tagged treebank exists, the linguist’s job seems to be over for that language. However, linguist’s job should not be an endless production of grammatical descriptions of a language. With a workable model of a language captured in the parsers, the linguist can turn to: –Improving the formalism for representing the language. –Going beyong syntax: using the functional syntactic parsers as tools to support the exploration of other areas of language: Propositional content, and how to extract it from syntactically annotated text, Thematic development of text, structure of dialogue, etc.

Are Linguists Dinosaurs? In summary, the advent of statistical parsers should not be seen as the end of the linguist’s work, but rather as the end of one phase, and the opportunity to move on into the study of meaning with firm support from below.