Presentation on theme: "By Rohana Mahmud (NLP week 1-2)"— Presentation transcript:
1 By Rohana Mahmud (NLP week 1-2) Communication & NLByRohana Mahmud(NLP week 1-2)
2 Study of Language Language: Written: Long-term record of knowledge from one generation to anotherSpoken: primary mean of coordinating day-to-day behavior with othersNatural (eg. Malay, English) vs. Artificial (Java, Prolog, Coding)CommunicationUse sign / natural language/ body languageSender and ReceiverStudied in several disciplines:Linguist: structure of languagePsychoLinguist: the process of human language production and comprehensionPhilospher: how words can mean anything & how they identify object in the world, what it means to have belief, goals and intention, cognitive capabilities relate to languageCL: to develop a computational theory of Language (using the notions of algorithm & data structure from CS)
3 History Speech & Language Processing: overlapping fields: Computational Linguistics (Linguistics)Natural Language Processing (CS)Speech Recognition (EE)Computational Psycholinguistics (Psychology)
4 History 1940s & 1950s – Foundational Insights: the automaton and probabilistic/ information-theoretic models (Turing1936, Shannon 1948)Formal language theory (chomsky 1956)Probabilistic algorithm: 2 Camps- Symbolic and Stochastic: 4 paradigms – Stochastic, Logic-based, NLU (SHRDLU Winograd, LUNAR Woods), Discourse Modelling (grosz) & Believe-Desire-IntentionCurrent: The Field comes together – probabilistic & data model, increase in speed and memory of computers & applied to Augmentative and Alternative Communication(AAC) and the rise of Web, need for language-based IR & IE.
5 Goal Scientific goal: Cognitive science – interdisciplinary research Technological/practical goal - NLP revolutionize the computers are used Computer that understand NL could access all information (human knowledge)NL interfaces to computers: allow complex system to be accessible to everyone. More flexible and intelligent
6 Speech and Language Processing Understand human: speech recognition & NLU (lip-reading)Communicating with human: NLG & Speech SynthesisInformation retrieval, information extraction, inference (draw conclusions based on facts)Spelling corrections, grammar checking, machine translationData processing vs. language processing (knowledge of language – what it means to be a word)
7 Application of NLUIt represents the meaning of sentences in some representation language that can be used later for further processingText-based applicationsWritten text processing (books,newpaper, reports, manual, , sms) = reading-based tasksSearching/finding from database of textExtracting information from textTranslating documentsSummarizing texts for certain purposeStory understanding
8 Application of NLUDialogue-based applications – involve human-machine communication (spoken / keyboard/mouse/ recognizer)Q&A systems, eg. Query databaseAutomated customer service (phone)Tutoring systems (interaction with students)Spoken language control of machineGeneral cooperative problem-solving systemSpeech recognition <> Language understanding system (only identify the word spoken from a given speech signal, not – how words are used to communicate)Discuss ELIZA system
9 ELIZA systemMid-1960s, MIT, a Therapist (system) & patient (user), Weizenbaum, 1966Algorithm:Has a Dbase of particular words (keywords)For each keyword -> store an integer, a pattern to match against the input and a specification of the outputGiven Sentence(S), find a keyword in S whose pattern matches SIf > 1 keyword, pick the one with highest integer valueUse the output specification that is associated with this keyword to generate next sentenceIf there are No keywords, generate an innocuous continuation statement, eg: Tell me more, Go on. (figure 1.2, 1.3 Allen)
10 Representations and Understanding Computing a representation of the meaning of sentences and texts (Notion of representation)Why can’t use the sentence itself as a representation of its meaning? Most words have multiple meanings (Senses). eg. Cook, bank, still (verb or noun), I made her duck. I saw a man in the park with a telescopeThus, ambiguity inhibit system from making the appropriate inferences needed to model understanding (need to resolve or disambiguate: eg. Use Lexical disambiguation: POS, word-sense disambiguation)A program must explicitly consider each senses of a word to understand a sentence
11 Represent meaning: must have a more precise language Mathematics & Logic and the use of formally specified representation languages (formal language) – notion of an atomic symbolUseful representation languages have 2 properties:Precise and unambiguousCapture the intuitive structure of the natural language sentences that it represents
12 Models and AlgorithmToolkit: state machines, formal rule systems, logic, probability theory, machine learningStates, transitions among states and inpur representationBasic procedural models: deterministic, Non-Deterministic finite-state automata, finite-state transducer -> weighted automata, markov model, HMMFormal rules: Regular grammar, regular relations, context free grammar (phonology, morphology, syntax)Involve a search through a state of spaces representing by hypotheses about an input : depth first, best-first and A* searchLogic – first order logic and predicate calculus, semantic network and conceptual dependency – logical representation (semantic, pragmatic and discourse)
13 Bibliography ACL (Association for CL) / EACL COLING (int conference of CL)Applied NLPWorkshop on Human Language TechnologyJournal: CL & NLEIEEE ICASSP: Acoustic, Speech and Signal ProcessingIEEE Transactions on Pattern Analysis and Machine IntelligenceIJCAI: Int Joint Conference on AIJournal: AI, Computational Intelligence, Cognitive Science
14 TUTORIAL – WEEK 2 Submit your first week Tutorial Presents your findingsDefinitions (NLP/NLG/NLU) (CL) (LE)History?Key works / research done in the areaReferences
15 Tutorial 2 - task Practical Hands-on with ELIZA system or similar systemSoftware?Algorithm?Domain?