Introduction to CL Session 1: 7/08/2011. What is computational linguistics? Processing natural language text by computers for practical applications.
Published byModified over 5 years ago
Presentation on theme: "Introduction to CL Session 1: 7/08/2011. What is computational linguistics? Processing natural language text by computers for practical applications."— Presentation transcript:
What is computational linguistics? Processing natural language text by computers for practical applications ... or linguistic research Among practical applications Sometimes the computer only needs to classify or transform the text ... but sometimes it needs to “understand” Ex: Watson: winner of ‘Jeopardy’ CL vs. NLP (natural language processing)
NLP applications Automatic speech recognition (ASR): speech text Machine translation (MT): L1 L2 Information retrieval (IR): Query + documents a subset of doc Information extraction (IE): document “database”
Other Applications Call Center Spam filter Spell checker Sentiment analysis: product reviews Bio-NLP: processing clinical data ….
Basic NLP tasks: Shallow processing Tokenization: – He visited New York in 2003. Morphological analysis: – visited visit + -ed Part-of-speech tagging – He/Pron visited/V New/?? York/N in/Prep 2003/CD Name-entity tagging – He visited [LOCATION New York] in [YEAR 2003] Chunking – [NP He] [V visited] [NP New York] in [NP 2003]
Basic NLP tasks: Deep processing Parsing – (S (NP (PRON he)) (VP (V visited) ….) Semantic analysis – Semantic tagging: [AGENT He] visited [DEST New York] …. – Meaning: visit (he, New-York) Discourse – Co-reference: “He” refers to “John” – Discourse structure Dialogue Generation
Ambiguity Phonological ambiguity: (ASR) – “too”, “two”, “to” – “ice cream” vs. “I scream” – “ta” in Mandarin: he, she, or it Morphological ambiguity: (morphological analysis) – unlockable: [[un-lock]-able] vs. [un-[lock-able]] Syntactic ambiguity: (parsing) – John saw a man with a telescope. – Time flies like an arrow.
Ambiguity (cont) Lexical ambiguity: (WSD) – Ex: “bank”, “saw”, “run” Semantic ambiguity: (semantic representation) – Ex: every boy loves his mother – Ex: John and Mary bought a house Discourse ambiguity: – Susan called Mary. She was sick. (coreference resolution) – It is pretty hot here. (intention resolution) Machine translation: – “brother”, “cousin”, “uncle”, etc.
Ambiguity resolution Rule-based or knowledge-based: – Parsing: I saw a man with a hat I saw a man with a telescope (in my hand) – WSD: “bank” – MT: “brother”, “cousin”, “uncle” Statistical approach: – Require training data – Build a statistical model – Knowledge and rules can be incorporated into the model as features etc.
Major approaches to NLP Rule-based approach Statistical approach – Supervised learning – Semi-supervised learning – Unsupervised learning
Supervised learning algorithms Hidden Markov Model (HMM) Decision tree Decision list Naïve Bayes Transformation-based Learning (TBL) Maximum Entropy (MaxEnt) Support Vector Machine (SVM) Conditional Random Field (CRF) …
Data Raw text: – Monolingual: English/Chinese/Arabic Gigawords – Parallel data: UN data, EuroParl Treebank: – Syntactic treebanks: a set of parse trees – Proposition Bank: – Discourse Treebank Dictionaries WordNet FrameNet …
The role of linguistics knowledge in NLP An NLP system is language-independent. Good or bad? – Good: it can be ported to many languages without any changes. – Bad: it cannot take advantage of properties of certain languages. How to incorporate (linguistic) knowledge in statistical systems? – the design of models – as features – as filters –…–… Building a treebank is an effective way.