Chapter 8. Learning of Gestures by Imitation in a Humanoid Robot in Imitation and Social Learning in Robots, Calinon and Billard. Course: Robots Learning from Humans Ryun, Seokyun Human Brain Function Laboratory Interdisciplinary Program in Neuroscience Seoul National University

Contents Introduction Experimental Set-up Data Processing –Preprocessing by principal component analysis (PCA) –Encoding in hidden Markov models (HMMs) –Recognition, Data retrieval, Imitation metrics Results and Performance of the System Discussion of the Model Conclusion Discussion (further) 2

Introduction Imitation abilities in human ?

Introduction Robot learning by imitation (robot programming by demonstration) explores novel means of implicitly teaching a robot new motor skills Advantages -Provide a natural, user-friendly means of implicitly programming the robot -Constrain the search space of motor learning by showing possible and/or optimal solutions Gesture recognition and reproduction Writing letters, arm trajectory… How to find a representation of the data that encapsulates only the key aspects of the gesture…?

Experimental Set-up

Training: by eight healthy volunteers (imitating six motions in video) Knocking on a door Raising a glass, drinking and putting it back on a table Waving goodbye Drawing a stylized alphabet letters A, B and C Data recording: by three x-sens motion sensors (location, joint angle) Sampling rate: 100 Hz by Color-based vision system (camera): arm tracking by markers

Data Processing

Preprocessing by principal component analysis (PCA) Determining the directions along which the variability of the data is maximal Minimize the statistical dependence across the data Dimensionality reduction Advantages Reduce noise Reduce dimensionality (faster processing) Simple representation

Data Processing Hidden Markov Models (HMMs) X = states, y = possible observations, a = transition probabilities, b = output probabilities Robust statistical algorithm in time-dependent sequential events (speech recognition, gesture recognition, etc.)

Data Processing Encoding in hidden Markov models (HMMs) A set of time series is used to train a HMM with I+F output variables. The parameters are expressed as a set of parameters (initial state distribution, state transition probabilities, means of output variables, standard deviation of the output variables, respectively)

Data Processing Training Transition probabilities and the observation distributions are estimated by expectation maximization (EM) algorithm Optimal number of state selection : Bayesian information criterion (BIC) L = likeligood of the fitted model n(p) = number of independent parameters in HMM T = number of observation data used in fitting the model

Data Processing Recognition Estimate the likelihood that the new signals could have been generated by one of the models Data retrieval Sequence of state is reconstructed by the Viterbi algorithm HMM states  location/angle sequences

Data Processing Imitation metrics Determining a metric of imitation performance Error estimation: by cost function In univariate case e = mean error of the observed data D’ compared to the dataset D u(t) = sequence of means associated with the sequence of states e’ = RMS error, x(t) = demonstration datasets, x’(t) = reproduction datasets

Data Processing Error estimation: In multidimensional case K = total dimensions, w = weights vector (importance of variable i) Sigma = standard deviation of each time-point/variable

Data Processing

Results and Performance of the System Recognition rate = 96 % (23 motions correct)

Results and Performance of the System

Discussion of the Model PCA: Making the HMM encoding more robust The advantage of encoding the signals in HMMs Better generalization of the data Robust to distortion in time (e.g. drinking timing: fast or slow) It would be interesting to extend the model to using asynchronous HMMs Joint probability of pairs of asynchronous sequences describing the same sequence of events (e.g. visual lip reading and audio signals) Visual + motor representation  alignment

Discussion of the Model Similarity with work in psychology and ethology Imitation using string parsing

Discussion of the Model Similarity with work in psychology and ethology Associative sequence learning (ASL)

Discussion of the Model Similarity with work in psychology and ethology Algebraic framework for the correspondence problem

Conclusion Implementation of an HMM-based system to encode, generalize, recognize and reproduce gestures, with representation of the data in visual and motor coordinates Link between theoretical concepts and practical applications Observed elements of a demonstration, and the organization of these elements, should be stochastically described to have a robust robot application

Discussion (further) Mimicking innate sensorimotor representation of human Visual markers  ?  corresponding body parts Self-feedback during imitation (tactile, proprioception)  Exercise HMM approach in imitation: similar to the function of cerebellum HMM in HMM subsets: for parallel and serial sequences Limitation: individualizing the gesture (based on numerous sensory feedback) (e.g. vestibular function, proprioceptive feedback, tactile feedback, movement tracking of own body parts by visual, sensory feedback)

Thank You