using common sense reasoning to create intelligent mobile applications software agents group mit...
TRANSCRIPT
![Page 1: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/1.jpg)
Using Common Sense Reasoning to Create Intelligent Mobile Applications
Software Agents GroupMIT Media Lab
![Page 2: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/2.jpg)
The PC vs. the Smart Phone
Current efforts to develop software for smart phones are based on taking PC applications
and making them smaller
![Page 3: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/3.jpg)
The PC vs. the Smart Phone
The PC has an email application The smart phone has a smaller one
Unlike the PC, text entry on smart phones is very difficult
![Page 4: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/4.jpg)
The PC vs. the Smart Phone
Because of the limited IO, the smart phone needs to be better at understanding the user
then the PC ever was
![Page 5: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/5.jpg)
The PC vs. the Smart Phone
Unlike the PC which stays on its desk, people carry their smart phones with them everywhere
![Page 6: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/6.jpg)
The PC vs. the Smart Phone
Because of this, the smart phone needs to be better at understanding the world then the PC
ever was
![Page 7: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/7.jpg)
The PC vs. the Smart Phone
To understand the user and understand the world, smart phone applications need something
that computers have always lacked, common sense reasoning
This is a problem we are trying to solve
![Page 8: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/8.jpg)
Introduction to Common Sense Reasoning
Over the past 3 years 12,000 people have entered common sense facts to a Web site called Open Mind
![Page 9: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/9.jpg)
Introduction to Common Sense Reasoning
Open Mind now contains over 650,000 common sense facts about the world
![Page 10: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/10.jpg)
Introduction to Common Sense Reasoning
Using this knowledge base we have created a semantic network of concepts and relationships called OMCSNet
![Page 11: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/11.jpg)
Using Common Sense Reasoning in Mobile Applications
We are currently using OMCSNet in two cell phone based applications
![Page 12: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/12.jpg)
Using Common Sense Reasoning in Mobile Applications
A language translation aid called GloBuddy 2
![Page 13: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/13.jpg)
Using Common Sense Reasoning in Mobile Applications
And a text entry system capable of contextually relevant word completion
![Page 14: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/14.jpg)
GloBuddy 2
![Page 15: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/15.jpg)
GloBuddy 2
When Traveling in a foreign country, people often rely on phrase books. However:
-They only cover a very limited number of common situations
-Even a common situation will quickly deviate from the book’s script
![Page 16: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/16.jpg)
GloBuddy 2
Translation Software and dictionaries currently exist for PDA devices
But users must write out the entire phrase they wish to translate, and look up each word individually
![Page 17: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/17.jpg)
GloBuddy 2 7 / 14
GloBuddy 2 will translate phrases for the user into English
But GloBuddy can go beyond simple translation and understand the context of the user’s situation
![Page 18: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/18.jpg)
GloBuddy 2
Instead of writing the entire phrase they wish to translate into a foreign language, GloBuddy asks the user to write the ideas they would like to talk
about.
GloBuddy then uses common sense reasoning to build a localized and contextually relevant
vocabulary of terms for the user
![Page 19: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/19.jpg)
GloBuddy 2
1. GloBuddy asks the user to describe the ideas they would like to talk about
2. GloBuddy then uses OpenMind and OMCSNet to build a localized vocabulary of terms
3. Contextually relevant words and phrases are displayed to the user for use in conversation
![Page 20: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/20.jpg)
GloBuddy 2
To return to our example in the introduction:
![Page 21: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/21.jpg)
GloBuddy 2
Because GloBuddy can intelligently expand on the user’s translation request:
The amount of text entry is significantly reduced
And Conversations are more fluid
![Page 22: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/22.jpg)
GloBuddy 2
There is a traditional trade off between a system’s breadth of information, and its depth of reasoning:
Breadth of Information
Dep
th o
f R
eason
ing
Phrase Book
Dictionary
Human Translator
![Page 23: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/23.jpg)
GloBuddy 2
GloBuddy 2 is able to avoid this tradeoff due to its extensive knowledge of common sense facts
Breadth of Information
Dep
th o
f R
eason
ing
Phrase Book
Dictionary
Human Translator
GloBuddy 2
![Page 24: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/24.jpg)
Word Completion
![Page 25: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/25.jpg)
Using Common Sense Reasoning in Mobile Applications
Current approaches to reducing text entry on Smartphones are limited
![Page 26: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/26.jpg)
Using Common Sense Reasoning in Mobile Applications
Using common sense reasoning, we can perform intelligent word completion based on the context of
the message
![Page 27: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/27.jpg)
Using Common Sense Reasoning in Mobile Applications
Some technical notes:
• Unlike GloBuddy, OMCSNet has to be stored locally
• This approach is best used in addition to classic word completion, rather then replacing it
![Page 28: Using Common Sense Reasoning to Create Intelligent Mobile Applications Software Agents Group MIT Media Lab](https://reader035.vdocuments.us/reader035/viewer/2022070409/56649e7c5503460f94b7e4c9/html5/thumbnails/28.jpg)
http://agents.media.mit.edu/projects/mobile