Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab

The PC vs. the Smart Phone Current efforts to develop software for smart phones are based on taking PC applications and making them smaller

The PC vs. the Smart Phone The PC has an applicationThe smart phone has a smaller one Unlike the PC, text entry on smart phones is very difficult

The PC vs. the Smart Phone Because of the limited IO, the smart phone needs to be better at understanding the user then the PC ever was

The PC vs. the Smart Phone Unlike the PC which stays on its desk, people carry their smart phones with them everywhere

The PC vs. the Smart Phone Because of this, the smart phone needs to be better at understanding the world then the PC ever was

The PC vs. the Smart Phone To understand the user and understand the world, smart phone applications need something that computers have always lacked, common sense reasoning This is a problem we are trying to solve

Introduction to Common Sense Reasoning Over the past 3 years 12,000 people have entered common sense facts to a Web site called Open Mind

Introduction to Common Sense Reasoning Open Mind now contains over 650,000 common sense facts about the world

Introduction to Common Sense Reasoning Using this knowledge base we have created a semantic network of concepts and relationships called OMCSNet

Using Common Sense Reasoning in Mobile Applications We are currently using OMCSNet in two cell phone based applications

Using Common Sense Reasoning in Mobile Applications A language translation aid called GloBuddy 2

Using Common Sense Reasoning in Mobile Applications And a text entry system capable of contextually relevant word completion

GloBuddy 2

When Traveling in a foreign country, people often rely on phrase books. However: -They only cover a very limited number of common situations -Even a common situation will quickly deviate from the book’s script

GloBuddy 2 Translation Software and dictionaries currently exist for PDA devices But users must write out the entire phrase they wish to translate, and look up each word individually

GloBuddy 2 7 / 14 GloBuddy 2 will translate phrases for the user into English But GloBuddy can go beyond simple translation and understand the context of the user’s situation

GloBuddy 2 Instead of writing the entire phrase they wish to translate into a foreign language, GloBuddy asks the user to write the ideas they would like to talk about. GloBuddy then uses common sense reasoning to build a localized and contextually relevant vocabulary of terms for the user

GloBuddy 2 1. GloBuddy asks the user to describe the ideas they would like to talk about 2. GloBuddy then uses OpenMind and OMCSNet to build a localized vocabulary of terms 3. Contextually relevant words and phrases are displayed to the user for use in conversation

GloBuddy 2 To return to our example in the introduction:

GloBuddy 2 Because GloBuddy can intelligently expand on the user’s translation request: The amount of text entry is significantly reduced And Conversations are more fluid

GloBuddy 2 There is a traditional trade off between a system’s breadth of information, and its depth of reasoning: Breadth of Information Depth of Reasoning Phrase Book Dictionary Human Translator

GloBuddy 2 GloBuddy 2 is able to avoid this tradeoff due to its extensive knowledge of common sense facts Breadth of Information Depth of Reasoning Phrase Book Dictionary Human Translator GloBuddy 2

Word Completion

Using Common Sense Reasoning in Mobile Applications Current approaches to reducing text entry on Smartphones are limited

Using Common Sense Reasoning in Mobile Applications Using common sense reasoning, we can perform intelligent word completion based on the context of the message

Using Common Sense Reasoning in Mobile Applications Some technical notes: Unlike GloBuddy, OMCSNet has to be stored locally This approach is best used in addition to classic word completion, rather then replacing it