NW Computational Intelligence Laboratory Experience-Based Surface-Discernment by a Quadruped Robot by Lars Holmstrom, Drew Toland, and George Lendaris Portland State University, Portland, OR

NW Computational Intelligence Laboratory Motivation for the Research Enhance our Understanding of Intelligence –Study aspects of how our minds work by implementing intelligent behavior in software and hardware Application –Create intelligent tools and algorithms to solve complex problems in a more “human- like” way

NW Computational Intelligence Laboratory Desirable Human-Like Abilities 1. Efficient transfer of knowledge from one problem domain to another

NW Computational Intelligence Laboratory Desirable Human-Like Abilities 2. Rapid Context Discernment (System Identification)

NW Computational Intelligence Laboratory Desirable Human-Like Abilities 3. The more knowledge one obtains, the more efficient one becomes at accessing and using that knowledge O(log n ) search for binary trees

NW Computational Intelligence Laboratory Experienced-Based Control Goal: to build into machines the ability to use past experience when – performing system identification, and – coming up with a good controller for a given situation To do so effectively and efficiently To do so in a “human-like” fashion

NW Computational Intelligence Laboratory Sony AIBO

NW Computational Intelligence Laboratory Available Data Vision IR sensors Accelerometers Joint positions

NW Computational Intelligence Laboratory AIBO Experience-Based Algorithm Goal: AIBO to change gait based on surface type – Identify change in surface Use only information available from joint actuators – Implement proper change to gait parameters Recall gaits from previously-experienced surfaces Desire to generalize to novel surfaces – Both tasks are to be based on past experiences with similar surfaces

NW Computational Intelligence Laboratory How Do We Endow AIBO with Experience? Base capability for walking behavior is provided by Sony (hardware) and Tekkotsu (software) Train AIBO to develop “good” gaits (gait parameters) for a selected set of distinct surfaces

NW Computational Intelligence Laboratory Genetic Algorithm Used to Develop/Learn Gaits Optimized for a balance of speed and sway on 4 different surfaces 1. Hardboard 2. Thin foam 3. Thin carpet 4. Shag carpet Each of these gaits performed significantly better than the default Tekkotsu gait (Chromosome)

NW Computational Intelligence Laboratory Context Discernment Our system now has Experience : Good GA gaits for a set of surfaces. Now, how do we get AIBO to recognize and then adapt to changes in surface qualities?

NW Computational Intelligence Laboratory Available Data Vision IR sensors Accelerometers Joint positions

NW Computational Intelligence Laboratory Observation: Issued Commands and Measured Motion Differ

NW Computational Intelligence Laboratory Are There Measurable Differences Between the Kinesthetic Responses for Different Surface Types?

NW Computational Intelligence Laboratory Complicating Attributes of Available Data for this Task Low sampling rate (~31Hz) Occasional dropped samples Large variance – Process noise? – Measurement noise? Time consuming to collect Non-stationarities – Surface Irregularities – Physical Dynamics of the AIBO

NW Computational Intelligence Laboratory Approach 1: Work in the Frequency Domain Smoothed Periodograms of Left Hip Joint, Thin-Foam Gait, on 4 Different Surfaces

NW Computational Intelligence Laboratory Approach 2: Work in the Time Domain Linear Forward Prediction Model For each of the 15 actuator signals, predict the current state of the actuator as a linear sum of the actuator’s past states. Find the mean squared error (MSE) of the predictions across all of the actuators at each time step. Using this procedure for a single gait/surface combination and set of actuator signals, we can generate a one-dimensional error signal

NW Computational Intelligence Laboratory Fitting the Models Fit one model on the data collected for each gait/surface combination Solution of the Normal Equations is performed to quickly find the unique and optimal model parameters for the given training data Above properties motivated use of linear models (for computational ease)

NW Computational Intelligence Laboratory Kinesthetic Experience (over all modeled gait/surface combinations) The Kinesthetic Experience is the (# gaits # surfaces ) dimensional signal indicating each model’s MSE as it unfolds over time In figure, error signal corres- ponding to the actual gait being used and surface being traversed is the minimum in each of these cases, indicating perfect classification

NW Computational Intelligence Laboratory Implemented Discernment of Surface Transition The algorithm can discern surface transitions in novel data within 2-4 seconds with 92% accuracy Accuracy increased as time increases

NW Computational Intelligence Laboratory The Same Approach Can Be Applied to Discerning Changes in Surface Incline Surface Discernment Surface Transition Discernment

NW Computational Intelligence Laboratory Experience Based Discernment and Control In Action

NW Computational Intelligence Laboratory Future Directions Use a set of gait/surface experiences as the base for discerning novel surface types – The Kinesthetic Experience acts as a parametric “description” of the surface being experienced Use the Kinesthetic Experience to generalize to new control policies to match new surfaces