1. Context-Specific Intention Awareness through Web Query
ICRA 2015 – NO.137
Nominated as best conference paper award, best student paper award, best cognitive robotics paper award.
Context-Specific Intention Awareness through Web Query
Rui Liu
Dr. Xiaoli Zhang
Jeremy Webb
Songpo Li
Colorado School of Mines, Colorado, USA
Intelligent Healthcare Robotics Lab
bio, cooperation

2. Colorado School of Mines, Colorado, USA
Introduction
CSIA
Explores environmental context to infer intention in robotic caregiving
Implicitly infer human intention by reducing user involvement
Challenges
Limited knowledge of Intention-environment correlation: source, cost
Lack of knowledge learning in new situations
WebIA: solution
Collect knowledge by web query
Update knowledge by Situations-driven
Colorado School of Mines, Colorado, USA
Intelligent Healthcare Robotics Lab
definition, important, challenging, to solve it.

3. Colorado School of Mines, Colorado, USA
WebIA Framework
Performance unsatisfied >> Learn >>Update
Colorado School of Mines, Colorado, USA
Intelligent Healthcare Robotics Lab
flow, update

4. Colorado School of Mines, Colorado, USA
WebIA
Answering 2 questions:
Is WebIA feasible ?
if feasible, how good it is
Colorado School of Mines, Colorado, USA
Intelligent Healthcare Robotics Lab
short mention 2 questions.

5. Colorado School of Mines, USA
Methodology: WebIA
Knowledge Representation
Intention-centered knowledge
Intention-object correlation
(affordance)
Intention-context correlation
(environment exploration)
Colorado School of Mines, USA
Intelligent Healthcare Robotics Lab

6. Colorado School of Mines, USA
Methodology: WebIA
Intention Inference Engine
Colorado School of Mines, USA
Intelligent Healthcare Robotics Lab

7. Colorado School of Mines, USA
Methodology: WebIA
Web Information Retrieval
Text from WikiHow
Parsing,
Pattern Matching
Dependency
Analysis
Correlations
NLP
Colorado School of Mines, USA
Intelligent Healthcare Robotics Lab

8. Colorado School of Mines, USA
Methodology: WebIA
Situation-driven Knowledge Update
to proactively update a robot’s knowledge to adapt to untrained new situations.
New situation detection: human-involved, untrained, not a synonym of trained objects
Knowledge update: establish the new intention-object correlations
Colorado School of Mines, USA
Intelligent Healthcare Robotics Lab

9. Colorado School of Mines, USA
Evaluation
Knowledge Acquiring: queried 1470 wikihow webpages, surveyed 120 volunteers.
Robot-involved experiments
The object cup has been identified by NAO. IA was performed based on the commonsense knowledge generated from WikiHow.
NAO points forward to the sink when the inferred intention is ‘wash’.
NAO waves its right hand to notify the caregiver when the inferred intention is ‘drink’.
The new situation ‘cup of coffee’ is detected; then NAO gains new knowledge by querying WikiHow to perform IA in this specific situation.
Colorado School of Mines, USA
Intelligent Healthcare Robotics Lab

10. Colorado School of Mines, USA
Evaluation
Cup-related Commonsense Knowledge from the Web
Accuracy:
Drink: 100%
Wash: 85%
Comparison of Knowledge from Web & Survey
Consistent
Colorado School of Mines, USA
Intelligent Healthcare Robotics Lab

11. Colorado School of Mines, USA
Evaluation
Situation-driven knowledge Update Evaluation
Intention Probability Adjustment from Old to New Situations
Detailed process in One IA Situation
New situation
before learning: 57.5%
after learning : 100%
Colorado School of Mines, USA
Intelligent Healthcare Robotics Lab

12. Colorado School of Mines, Colorado, USA
Conclusion
WebIA effectively learn the relation between human intention-environment and accurately inferred human intention in specific situations.
Feasible: WebIA has potential to query vast and valuable knowledge from the internet to assist human-robot interaction
Reasonable: Web knowledge is helpful in filling robots’ knowledge gap for new situation adaptation
Economic: WebIA reduces the training cost
Colorado School of Mines, Colorado, USA
Intelligent Healthcare Robotics Lab
effectively learn, accurately infer.

13. Colorado School of Mines, USA
Rui Liu
Xiaoli Zhang
Colorado School of Mines, USA
Intelligent Healthcare Robotics Lab

14. Colorado School of Mines, USA
Authors
Rui Liu, PhD Student
Interests: robot knowledge, computational linguistics and machine learning
ResearchGate:
Google Scholar:
Xiaoli Zhang, Assistant Professor
Interests: intelligent human-robot interaction, human intention awareness, robotics system design and control, haptics, and their applications in healthcare fields
Colorado School of Mines, USA
Intelligent Healthcare Robotics Lab

15. Context-Specific Intention Awareness through Web Query
Rui Liu1, Xiaoli Zhang1,Songpo Li1 and Jeremy Webb1
1Colorado School of Mines
Use web query to
establish a vast and
active context-related
knowledge for robot
to infer human
intention in both trained
and untrained environment.