A User study on Conversational Software Yan Ke

Intelligent Computer System Target：Use machine to reinvent human intelligence What is AI？-- No concrete definition

What is AI? Views of AI fall into four categories: Thinking humanly Thinking rationally Acting humanly Acting rationally

Acting humanly: Turing Test Turing (1950) "Computing machinery and intelligence": "Can machines think?" "Can machines behave intelligently?" Operational test for intelligent behavior: the Imitation Game Predicted that by 2020, a machine might have a 50% chance of fooling a lay person for 30 minutes

The main research area of AI Natural Language Processing Computer can correctly understand human languages For different language inputs, computer can efficiently summarize the abstract of the input information And can translate the input information using any required languages

Natural Language Processing (NLP)

Main steps of NLP Input： Understanding the input Looking for answer Voice recognition（out of scope） Understanding the input Partitioning the sentence (noun, verb, subjective, objective) Searching for the meaning in database Looking for answer Searching for appropriate answer related to the input

Basic Functions of NLP Question Answering: Using the correct grammar to answer certain questions Information Extraction Explain Translate

The Advanced Methods for NLP Key words Searching Sentiment Analysis Knowledge-based NLP NLP based on big data

Information Extraction

Sentiment Analysis

Translation

Parsing (in DCG) s --> np, vp. det -->[a]. det --> [an]. np --> det, noun. det --> [the]. np --> proper_noun. noun --> [apple]. vp --> v, ng. noun --> [orange]. vp --> v. proper_noun --> [john]. proper_noun --> [mary]. v --> [eats]. v --> [loves]. Eg. john eats an apple. proper_noun v det noun np np vp s

Syntax Parse Trees

Parsing & semantic analysis Rules: syntactic rules or semantic rules What component can be combined with what component? What is the result of the combination? Categories Syntactic categories: Verb, Noun, … Semantic categories: Person, Fruit, Apple, … Analyses Recognize the category of an element See how different elements can be combined into a sentence Problem: The choice is often not unique

Pragmatic analysis Practical usage of language: what a sentence means in practice Do you have time? How do you do? It is too cold to go outside! …

Some success stories - ELIZA young woman: Men are all alike. eliza: In what way? young woman: They're always bugging us about something specific or other. eliza: Can you think of a specific example? young woman: Well, my boyfriend made me come here. eliza: Your boyfriend made you come here? young woman: He says I'm depressed much of the time. eliza: I am sorry to hear you are depressed. young woman: It's true. I'm unhappy. eliza: Do you think coming here will help you not to be unhappy? young woman: I need some help, that much seems certain. eliza: What would it mean to you if you got some help? young woman: Perhaps I could learn to get along with my mother. eliza: Tell me more about your family. young woman: My mother takes care of me.

ELIZA – Cont’d eliza: Who else in your family takes care of you? young woman: My father. eliza: Your father. young woman: You are like my father in some ways. eliza: What resemblance do you see? young woman: You are not very aggressive but I think you don't want me to notice that. eliza: What makes you think I am not very aggressive? young woman: You don't argue with me. eliza: Why do you think I don't argue with you? young woman: You are afraid of me. eliza: Does it please you to believe I am afraid of you? young woman: My father is afraid of everybody. eliza: What else comes to mind when you think of your father? young woman: Bullies. eliza: Does that have anything to do with the fact that your boyfriend made you come here?

Methods used in Eliza Find a trigger in the input of the user (e.g. father) Evoke a possible candidate pattern (e.g. family or mother) (~limited parsing) Compose a sentence by filling in the slots of the pattern (picking some elements from the user input) If no appropriate pattern is found, ask a general question, possibly related to the user input

Statistical language modeling Goal: create a statistical model so that one can calculate the probability of a sequence of tokens s = w1, w2,…, wn in a language. General approach: s Training corpus Probabilities of the observed elements P(s)

Prob. of a sequence of words Elements to be estimated: If hi is too long, one cannot observe (hi, wi) in the training corpus, and (hi, wi) is hard generalize Solution: limit the length of hi

Neural Networks

Assignment 1 (15% of the final mark) Find ten conversational software agents (chat robots) at your own convenience, chat with them with 5-10mins with at least 10 meaningful questions for each group member, and write an essay about: Which chatbots do you test on (can be iphone siri (5,6,7,8,X, XS, XS Max), Andoid, 淘宝customer service, uc, qq, or anything you can find across the web) Write down each question that you ask and the answer from the software agent. Evaluate your questions hardness and the answers relevance. Your question can be: Obvious Simple Neutral Difficult to answer Confusing question The evaluation of the software answer can be: Expert answer that beyond my expectation Good answer Irrelevant answer (It is not what I ask for) Fail to answer (The software says it does not know)

Assignment 1 (15% of the final mark) Talk about your user experience (心得体会) Which one is the best software that you have tested? What are advantages of the ‘best’ conversational software comparing with the rest pieces? What are the possible improvement can be proposed.