1. Chapter 10 Natural Language Processing Xiu-jun GONG (Ph. D) School of Computer Science and Technology, Tianjin University gongxj@tju.edu.cn http://cs.tju.edu.cn/faculties/gongxj/course/ai/

2. Turing's test  Is a machine is intelligent or not? If a machine could carry on a conversation (over a teletype) that was indistinguishable from a conversation with a human being, then the machine could be called "intelligent." Understand the language is the essential step for conservation

3. Language  We can define a language formally as: a set of symbols (“alphabet”) a set of rules to combine those symbols  This mathematical definition covers many classes of languages, not just human language.

4. Java: An artificial (formal) language  fixed set of basic symbols: public, static, for, while, {, }…  fixed syntax for symbol combination public static void main (String[] args) { for(int i = 0; i < args.length; i++) { … } }

5. Natural language  Often much more complicated than an artificial language. Some people view NL as a special kind of formal language as well.  It does conform to the formal definition: there are symbols there are modes of combination  However, there are many levels at which these symbols and rules are defined.

6. Levels of analysis in Natural language (1)  Acoustic properties (phonetics: 语音学 ) defines a basic set of sounds in terms of their features studies the combination of these phonemes  Higher-order acoustic features (phonology: 语 音体系 ) how combinations of phonemes combine into larger units, with supra-segmental features such as intonation.

7. Levels of analysis in Natural language (2)  Word formation (morphology: 形态学 ) combines morphemes into words  Combination into longer units in a structure- dependent way (syntax) “legal” word combinations in a language recursive phrasal combination  Interpretation (semantics): of words (lexical semantics) of longer units (sentential/propositional semantics)  Interpretation in context (pragmatics: 语用论 )

8. Natural Language Processing  Studies language at all its levels. phonology, morphology, syntax, semantics… focuses on process computational methods to understand and generate human language  Often, NLP is called computational linguistics

9. Task  Language Modeling  Machine Translation  Information Extraction/Retrieval  Named entity recognition  Automatic summarization  Speech recognition  Question Answering

10. Tools for NLP  OpenNLP http://opennlp.sourceforge.net/  Stanford NLP http://nlp.stanford.edu/links/statnlp.html  NLP Toolsuite http://www.julielab.de/Resources/Software/NL P_Tools.html http://www.julielab.de/Resources/Software/NL P_Tools.html  Resource maintained by Kita http://www-a2k.is.tokushima- u.ac.jp/member/kita/NLP/nlp_tools.html http://www-a2k.is.tokushima- u.ac.jp/member/kita/NLP/nlp_tools.html