1. Beating Common Sense into Interactive Applications Henry Lieberman, Hugo Liu, Push Singh, Barbara Barry AI Magazine, Winter 2004 As (mis-)interpreted by Peter Clark for Boeing KR Group

2. Introduction Claim: Commonsense applications are closer than you think Problems with CommonSense (CS) applications: –Even large KBs have sparse coverage –Inference is unreliable

3. Their Common Sense KB: Open Mind Common Sense (OMCS) 750k NL assertions from 15k contributors ConceptNet: A semantic net built from these –20 link types

5. Against Question-Answering… Question answering is a bad CS domain: –User expects a direct answer to all his/her questions –System has to be right (almost) all of the time –Got to be fast (few seconds) Alternative: intelligent interfaces –Assists user when it can –“fail soft” - user can ignore it if he/she wants –But: Is yet another paperclip?

6. 1. ARIA: Annotation and Retrieval Integration Agent Helps annotate photos, and find photos –Similar to Thesaurus search –Photos are annotated with keywords –a. People, places and events are recognized in text –b. Use the semantic net to find “close” photos to text –Text also adds to the net (system learns) “My sister’s name is Mary” → “Joe –sister→Mary”

7. 2. Detecting Moods (“affect”) in Text Approach: –Mood keyword (e.g., “sad”) → mine a “small society of linguistic models of effect” from the KB (=?) Applications: –Empathy Buddy: (purpose=?) –Summarizing a collection of reviews about a topic “My wife left me; she took the kids and the dog”   

8. 3. Cinematic Commonsense: Video Capture and Editing Videographer shoots, adds NL annotations –E.g., “a street artist is painting a painting” Send annotations to KB for elaboration –“after painting, you clean the brushes” –“during painting, you might get paint on your hands” Elaborations –suggest new shots for the videographer –Also are stored for improved retrieval –Can help order shots into temporal/causal (isn’t temporal ordering already done?) But: need more complex story understanding to create effective suggestions for the filmmaker.

9. 4. Common Sense Storytelling: StoryIllustrator Continuosly retreive photos relevant to user’s typing Use Yahoo image search, not annotations, for Web images CSK for query expansion, E.g., “baby” ↔ “milk”

10. 5. Common Sense Storytelling: OMAdventure E.g., in kitchen → what do you find in kitchen? → Other associated locations? E.g., oven → what can you do with an oven? Hence oven, cooking are “moves” for user. Associated locations are “exits”. User can add objects (e.g., “blender”) → extend KB (“blenders are in kitchens”) Generates dungeons-and-dragons game on the fly

11. 7. Common Sense Storytelling: StoryFighter System and user take turns to contribute lines to a story to get from A to B, e.g., –“John is sleepy” (start) –“John is in prison” (end) Must avoid “taboo” words (e.g., “arrest”) CSK deduces consequences of an event –“If you commit a crime, you might go to jail” CSK also picks obvious taboo words

13. 8. Topic Spotting Task: Given speech, identify situation E.g., “fries”, “lunch”, “Styrofoam” → “eating in a fast-food restaurant” Use Bayesian inference + ConceptNet Used in collaborative storytelling with kids –Computer starts the story –Kid continues –Computer can’t fully understand kid’s speech, but can at least identify the topic → generate plausible continuation E.g., “bedroom” → “Jane’s parents walked into the bedroom while she was hiding under the bed”

14. 9. Globuddy: A Tourist Phrasebook Type in your situation –“I’ve just been arrested” It retrieves and translates associated CS (?) –“If you are arrested, you should call a lawyer” –“Bail is a payment that allows an accused person to get out of jail until a trial”

16. 10. Predictive typing/phrase completion E.g., for a cellphone keyboard Use ConceptNet to find next word that “makes sense” –E.g., “train st” → “train station”

17. 11. Search: GOOSE and Reformulator as Google adjuncts Infer user’s search goals and add keywords,e,g,: –“my cat is sick” → “Did you mean to look for veterinarians?” Currently interactive. Later, will suggest better query.

18. 12. Semantic Web Given user’s goals, find services that might accomplish subgoals, e.g., –“Schedule a doctor’s appt” → look up directory of doctors, check reputation, geographic lookup, lookup schedules, etc.

19. 13. Knowledge Acquisition Criticism: Many OpenMind sentences are decontextualized –“At a wedding, bride and groom exchange rings” is culturally specific → develop a prompt-based interface to have user’s make context explicit.

20. Reflections Logic: What inferences are possible Commonsense: What inferences are plausible Qn: How well does OpenMind support this? E.g., –“People live in houses” –“Things fall down rather than up” –“Acid irritates skin”

21. Same for our own database… “There is a rocket” + = ?

22. Reflections (cont): Limitations Spottiness of subject coverage in OpenMind Inference is unreliable → reluctant to use it –Need new inference methods –E.g., “interleave context-sensitive inference with retrieval in a breadth-first manner” CS suggestions may be distracting –But trials suggest otherwise (people tolerate wrong but plausible suggestions better than stupid ones)

23. Some additional thoughts… Domain-specific vs. domain-general applications –Domain-specific – how much CS is needed? CycSecure Oil exploration etc. –Domain-general – still need task-specific algorithm –Unusual to find a domain- and task-general application “Scenario completion” is a good task –newswire, incident reports, etc.