Introduction Chapter 1 Foundations of statistical natural language processing

NLP and Statistical Approach Why many people are adopting a statistical approach to natural language processing? How one should approach this approach? –We will begin with discussion of some philosophical themes and leading ideas

Approaches to language Between 1960 and 1985 most of linguistics, Psychology, Artificial Intelligence and NLP was dominated by Rationalist approach “Significant part of the knowledge in the human mind is not derived by the senses but is fixed in advance, presumably by genetic inheritance”

Rationalist approach Dominated the field due to widespread acceptance of arguments by Noam Chomsky Argument: “Problem of poverty of stimulus” Difficult to see how children can learn something complex as natural language from limited input Questions?

Empiricist Approach Also begins with cognitive abilities point Difference between approaches is in terms of degree of belief “ Mind does not begin with detailed sets of principles/procedures for various components of language and things like morphological structure, case marking etc”. Baby’s brain begins with general operations of associations, pattern recognition, and generalization

Empiricist approach to NLP suggest that “ We can learn complicated and extensive language structures by specifying appropriate general language model” “and then using Statistical, Pattern Recognition and Machine Learning models to a large amount of language use”

SNLP People cannot work from observing a large amount of language usage –Instead simple ‘texts’ are used –A body of text is called Corpus (pl: Corpora) –Empiricist corpus-based approach is seen in American Structuralists (Zelling Harris ) Language’s structure can be discovered automatically using corpus

---- Chomskyan linguistics seeks to describe language model of human mind (I- language), for which texts (E-language) provide indirect evidence Empiricist approaches describe E-language as it ACTUALLY occurs –Chomsky postulates Linguistics competence Linguistic performance

Chomskyan linguistics depends on categorical principles –‘Do’ or ‘Do not’ satisfy –Same as American Structuralism –Categorical judgment of rare type of sentences Our approach would be inspired of Statistical NLP draws from work of Shannon –Assign probabilities to linguistic events to decided which sentences are ‘usual’ and which are ‘unusual’ –Associations and preferences occur in totality of language use

Scientific Content Questions that linguistics should answer –What kind of things do people say? –What do these things say/ask/request about the world. Key point: How knowledge of language is acquired by humans, and how they actually understand and generate sentences in real time

Competence Grammar –Said to underlie the language –Generative approach in speaker’s head It suggests that there is a set of sentences -Grammatical Sentences- and other strings which are ungrammatical The concept of grammaticality –Judged on how sentence is structurally well formed –Not according to what people say or semantically anomalous e.g. “ Colorless green ideas sleep furiously”

Syntactic grammaticality is a binary choice –Native speaker normally produces grammatical sentences Two points –Binary choice is plausible for simple sentences but for complex it may be farfetched –Non native speakers speak something grammatical but somehow odd. “ In addition to this, she insisted that women were regarded as a different existence from men unfairly ”

Non-categorical phenomena in language Categorical view of language may be sufficient for many purposes but has its limitations Frequency based analysis is required To see non-categorical phenomena change in the language should be studied –e.g. ‘While’ (noun)  time “Take a while” –While (Complementizer) “While you were out” After analyzing frequency, category should be reanalyzed

‘near’Adjective/Preposition –We will review that decision in the near future –He lives near the station –We live nearer the water than you thought Grammatically adjectives and nouns do not take direct object but preposition –‘convenient for people’ Comparative form is like adjective/adverbs Blending and Language change Kind of, sort of –We are kind of hungry

Summing up Few attempts to use statistical NLP for explaining complex linguistics phenomena –This new way of looking at language may be able to account for things such as non categorical phenomenon and language change Supportive argument “human cognition is probabilistic and that language must be too ”