PETRA 2014 An Interactive Learning and Adaptation Framework for Socially Assistive Robotics: An Interactive Reinforcement Learning Approach Konstantinos.

Slides:



Advertisements
Similar presentations
SEMANTICALLY RICH EDUCATIONAL WORD GAMES ENHANCED BY SOFTWARE AGENTS Boyan Bontchev, Sergey Varbanov, Dessislava Vassileva INFOS 2011 Rzeszów - Polańczyk,
Advertisements

Breakout session B questions. Research directions/areas Multi-modal perception cognition and interaction Learning, adaptation and imitation Design and.
Visual Data Mining: Concepts, Frameworks and Algorithm Development Student: Fasheng Qiu Instructor: Dr. Yingshu Li.
Background Reinforcement Learning (RL) agents learn to do tasks by iteratively performing actions in the world and using resulting experiences to decide.
A Generic Framework for Handling Uncertain Data with Local Correlations Xiang Lian and Lei Chen Department of Computer Science and Engineering The Hong.
Bayesian Reinforcement Learning with Gaussian Processes Huanren Zhang Electrical and Computer Engineering Purdue University.
Apprenticeship Learning for the Dynamics Model Overview  Challenges in reinforcement learning for complex physical systems such as helicopters:  Data.
ILMDA: Intelligent Learning Materials Delivery Agents Goal The ILMDA project is aimed at building an intelligent agent with machine learning capabilities.
University of Jyväskylä – Department of Mathematical Information Technology Computer Science Teacher Education ICNEE 2004 Topic Case Driven Approach for.
The Computer Science Course at Omar Al-Mukhtar University, Libya The Computer Science Course at Omar Al-Mukhtar University, Libya User-Centered Design.
Curious Characters in Multiuser Games: A Study in Motivated Reinforcement Learning for Creative Behavior Policies * Mary Lou Maher University of Sydney.
8/26/20151 Kick-off meeting 15-16/12/2008 Objectives and outcome Two year project C. Kourkoumelis, UoA.
Maestro: A Computer Tutor for Writers Kurt Rowley, Ph.D. Maestro Principal Investigator 'They [students] seem to be less distressed.
Designing Software for Elderly Suffering from Cognitive Impairments M. PINO, M. BOULAY, AS. RIGAUD Research team EA 4468 Paris Descartes University, Broca.
REINFORCEMENT LEARNING LEARNING TO PERFORM BEST ACTIONS BY REWARDS Tayfun Gürel.
Efficient Interaction Strategies for Adaptive Reminding Julie S. Weber & Martha E. Pollack Adaptive Reminder Generation SignalingIntended Approach Learning.
Reinforcement Learning for Spoken Dialogue Systems: Comparing Strengths & Weaknesses for Practical Deployment Tim Paek Microsoft Research Dialogue on Dialogues.
A Smartphone Application for Improving Powered Seat Functions Usage: A Preliminary Test Yu-Kuang Wu, MS PT, Hsin-Yi Liu, MS PT, Josh Brown, BS, Annmarie.
Model-based Bayesian Reinforcement Learning in Partially Observable Domains by Pascal Poupart and Nikos Vlassis (2008 International Symposium on Artificial.
Grenoble Informatics Laboratory (LIG) HCI Engineering Research Group (IIHM) Helping the design of Mixed Systems Céline Coutrix PhD Supervisor: Laurence.
Chapter 16. Basal Ganglia Models for Autonomous Behavior Learning in Creating Brain-Like Intelligence, Sendhoff et al. Course: Robots Learning from Humans.
Design Principles for Creating Human-Shapable Agents W. Bradley Knox, Ian Fasel, and Peter Stone The University of Texas at Austin Department of Computer.
PhD CONCEPT Exploring Digital Maps as Communication Tools for eParticipation - On an example of Urban Planning PhD concept, Beate Weninger,
Streamflow - Programming Model for Data Streaming in Scientific Workflows Chathura Herath.
Reinforcement Learning
REU 2004 Computer Science and Engineering Department The University of Texas at Arlington Research Experiences for Undergraduates in Distributed Rational.
Curiosity-Driven Exploration with Planning Trajectories Tyler Streeter PhD Student, Human Computer Interaction Iowa State University
Human-Computer Interaction at CMU Jodi Forlizzi Jason Hong.
Instructor Training Los Angeles County Sheriff CERT Level 1.
W. L. Johnson and J. T Rickel. “Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments,” International Journal of Artificial.
Name of presentation Integrating ICT: Part 1. Learning Objectives  increase our understanding of ICT integration  increase our ability to use ICT in.
REU 2007 Computer Science and Engineering Department The University of Texas at Arlington Research Experiences for Undergraduates in Information Processing.
Reinforcement Learning for Mapping Instructions to Actions S.R.K. Branavan, Harr Chen, Luke S. Zettlemoyer, Regina Barzilay Computer Science and Artificial.
REU 2009 Computer Science and Engineering Department The University of Texas at Arlington Research Experiences for Undergraduates in Information Processing.
1 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | GUI.
Human Performance Enhancement System Park Young Ho Dept. of Nuclear & Quantum Engineering Korea Advanced Institute of Science and Technology December 23.
Marilyn Wolf1 With contributions from:
MoMat: Mobile Manipulation
Presented by: Saurav Kumar Bengani
Online Multiscale Dynamic Topic Models
Towards Standards for Goal-Based Operations
Context-Specific Intention Awareness through Web Query
LU4 Promoting Learning & Continuous Development Opportunities
CS b659: Intelligent Robotics
UF Graduate Linguistics Society Seminar Series
Human-exoskeleton combined model
Context-Specific Intention Awareness through Web Query
Petar Kormushev, Sylvain Calinon and Darwin G. Caldwell
HI 5354 – Cognitive Engineering
Yu-Guang Chen1,2, Wan-Yu Wen1, Tao Wang2,
th IEEE International Conference on Sensing, Communication and Networking Online Incentive Mechanism for Mobile Crowdsourcing based on Two-tiered.
Strategies and Techniques
Joelle Pineau: General info
Using Instructional Software
An Adaptive Middleware for Supporting Time-Critical Event Response
Reinforcement Learning
Smart Learning concepts to enhance SMART Universities in Africa
یادگیری تقویتی Reinforcement Learning
Drill & Practice Programs
HERACLEIA Human Centered Computing Lab,
Reinforcement Learning
Data-Driven Decision-Making
Actively Learning Ontology Matching via User Interaction
Intelligent Process Automation in Audit
BCI Research at the ISRC, University of Ulster N. Ireland, UK
Department of Computer Science Ben-Gurion University
Irene-Angelica Chounta Senior Researcher
Urban Infrastructure: Analysis and Modeling for Their Optimal Management and Operation NSF Workshop NSF Award #: Nada Marie Anid, Ph.D. Professor.
Morteza Kheirkhah University College London
A Deep Reinforcement Learning Approach to Traffic Management
Presentation transcript:

PETRA 2014 An Interactive Learning and Adaptation Framework for Socially Assistive Robotics: An Interactive Reinforcement Learning Approach Konstantinos Tsiakas, Ph.D. Candidate HERACLEIA Human-Centered Computing Laboratory Computer Science and Engineering Department University of Texas at Arlington, USA Institute of Informatics and Telecommunications National Center for Scientific Research, Demokritos, Greece December 2016

Outline Introduction Socially Assistive Robotics PETRA 2014 Outline Introduction Socially Assistive Robotics Methodology and Approach Preliminary Work Issues and Challenges Robot Behavior Modeling Use Case Preliminary Experiments Motivation Interactive Learning and Adaptation Framework Description Experimental Results Conclusion Ongoing Work Research Plan Discussion

Socially Assistive Robotics PETRA 2014 Socially Assistive Robotics Socially Assistive Robotics (SAR) is an HRI research area that studies how robots can assist users during a cognitive or physical task through social interaction (Feil & Mataric 2005) Providing an adaptive and personalized interaction maximizes user engagement, motivation and thus performance (Fasola & Mataric 2010) Adaptation and personalization can be achieved through task parameters adjustment (e.g., difficulty) and through robot behavior (feedback, encouragement, etc.) A key challenge is to develop proper computational methods for personalization and adaptation in real-time, ensuring efficiency and safety

Socially Assistive Robotics PETRA 2014 Socially Assistive Robotics

Methodology and Approach PETRA 2014 Methodology and Approach Reinforcement Learning Reinforcement Learning (RL) provides an appropriate framework for behavior modeling and optimization An RL agent learns behavior through trial-and-error interactions with its environment Markov Decision Processes (MDP) MDP is defined as a tuple <S, A, T, R> Optimal policy π: S → A π* = argmaxaQ(s,a) environment action reward state

Preliminary Work PETRA 2014 K. Tsiakas et. al,A Multimodal Adaptive Rehabilitation Session Manager for Physical Exercising", In PETRA 2015.

Robot Behavior Modeling PETRA 2014 Robot Behavior Modeling

PETRA 2014 Learning Experiments

PETRA 2014 Transfer Experiments

Motivation Issues and Challenges PETRA 2014 Motivation Issues and Challenges RL agents require many iterations and interaction data Online learning methods (e.g., exploration) may be inappropriate for HRI Human → Dynamic environment Long-term adaptation Short-term adaptation Co-adaptation Interactive Learning and Adaptation Framework Integrates Interactive Reinforcement Learning techniques to facilitate the adaptation of a learning agent to each user, ensuring efficacy and safety Allows the agent to dynamically adapt during the interaction Allows human users to shape agent’s policy through feedback and guidance (implicitly and/or explicitly) Supports the participation of an external supervisor that can guide the system during its early interaction steps

Interactive Learning and Adaptation PETRA 2014 Interactive Learning and Adaptation K. Tsiakas et. al “Adaptive Robot Assisted Therapy using Interactive Reinforcement Learning", In ICSR 2016.

Interactive Learning and Adaptation PETRA 2014 Interactive Learning and Adaptation Teaching on a budget (Torrey et al. 2013) Q-augmentation (Chanel et al. 2008)

PETRA 2014 Experimental Results

PETRA 2014 Ongoing Work

PETRA 2014 Ongoing Work DEMO: https://youtu.be/TfL3BITWzG0

Research Plan Thesis Proposal PETRA 2014 Research Plan Thesis Proposal Publish HRI dataset and analysis (HRI conference) EEG analysis Correlation between performance, difficulty, feedback and engagement values → adaptive and personalized training Reinforcement Learning Train offline with learned user model Repeat interactive learning and adaptation experiments Develop supervisor GUI Visualize state uncertainty and importance -- Active Learning HRI User study -- Adaptive vs. Non-Adaptive Training Session Journal of Social Robotics

PETRA 2014 Related Publications K. Tsiakas, M. Abujelala, A. Lioulemes, F. Makedon, "An Intelligent Interactive Learning and Adaptation Framework for Robot-Based Vocational Training", in IEEE Symposium Series in Computational Intelligence,C2RAT, SSCI 2016 K. Tsiakas, M. Dagioglou, V. Karkaletsis, F. Makedon., "Adaptive Robot Assisted Therapy using Interactive Reinforcement Learning", in International Conference on Social Robotics, ICSR 2016 K. Tsiakas, M. Papakostas, B. Cheeba, D. Ebert,. V. Karkaletsis, F. Makedon., "An Interactive Learning and Adaptation Framework for Adaptive Robot Assisted Therapy", In Proceedings of the 9th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA 2016. K.Tsiakas, C. Abellanoza, F. Makedon, "Interactive Learning and Adaptation for Robot Assisted Therapy for People with Dementia", In Proceedings of the 9th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA 2016. BEST STUDENT PAPER AWARD K. Tsiakas, "Facilitating Safe Adaptation of Learning Agents using Interactive Reinforcement Learning", Student Consortium, Intelligent User Interfaces, ACM IUI 2016 K. Tsiakas, M. Huber and F. Makedon. "A Multimodal Adaptive Rehabilitation Session Manager for Physical Exercising", In Proceedings of the 8th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA 2015.

PETRA 2014 Questions Contact konstantinos.tsiakas@mavs.uta.edu