1. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA BECCA—A Brain Emulating Cognition and Control Architecture Brandon Rohrer Intelligent Systems, Robotics, and Cybernetics Group Sandia National Laboratories 2008 AAAI, BICA Workshop

2. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA Biases: What “biologically-inspired” means to me Self preservation and perpetuation is the implicit goal of biological systems. –If it is not, they don’t last very long Implies real-world interaction There is no a priori model of the environment –All categories and concepts are acquired Human cognition violates rationality, formal logics, optimality, universal concept definitions, strict ontologies, and strict perceptual categorizations –A BICA should not presuppose any of these Broad functional equivalence to humans is the goal of BICA research –All empirical data (neuroanatomy, cognitive neuroscience, experimental psychology, etc.) are rich sources for clues about how this might be achieved –Existing theories and constructs describing empirical data should be used with caution

3. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA BECCA—A Brain Emulating Cognition and Control Architecture Brandon Rohrer BECCA is a biomimetic approach to achieving human-like reasoning, perception, learning, and movement control in machines It has two core algorithms –S-Learning: A solution to some Dynamic Reinforcement Learning problems –Context-Based Similarity: A solution to the problem of semantically clustering symbols Given an ordered set of symbols, determine the semantic distance (similarity) between any two

4. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA BECCA Operational Diagram Sensor and control information are handled symbolically – are passed in “episodic” fashion, quantized in time, –are discretized in magnitude, –are treated categorically extrapolation and interpolation does not occur explicitly. Allows very general application

5. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA S-Learning Algorithm: A solution to some Dynamic RL problems S-Learning (sequence learning) –records observed sequences and uses them to make predictions about future events and control decisions Algorithm description –1. Learning At each iteration, record the longest novel sequence Reinforce the longest familiar sequence –2. Prediction Find sequences that begin with the most recent state(s) The remainder of those sequences constitute predictions –3. Control Evaluate each prediction for potential reward Select a prediction to re-enact Execute the same action sequence

6. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA S-Learning: Rotary robot Simulation of a one degree-of- freedom rotary pointer robot, –Sensor quantized in 10º increments –Movement by 10º increments S-Learning demonstrated the ability to learn and predict hard nonlinearities S-Learning performed optimally even in the presence of –Scrambled sensor conditions –Gain reversals –Stochastic movement errors –Random time delays No explicit model of the system was provided—its workings were discovered by S-Learning

7. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA S-Learning: Reaching simulation Two degree-of-freedom robot reaching simulation –Approximately human parameters used for inertia. movement characteristics, and sensing capabilities Robot learned to reach a fixed target at an arbitrary position in the plane Demonstrated generalization –Learning in one task was applied to a second task –This, despite the complete separation of the sensory representations of the two tasks No explicit model of the system was provided —its workings were discovered by S-Learning Early Mature Intermediate Click to play videos

8. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA S-Learning: Grasping simulation Three degree-of-freedom robot grasping simulation with rich sensors: –Coarse vision –Coarse position –Contact pressure Robot learned to reach a fixed target at an given position in the plane Learned to coordinate grasp with motion to grab target No explicit model of the system was provided —its workings were discovered by S-Learning Click to play video

9. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA Context-Based Similarity (CBS): A solution to semantic symbol clustering Underlying concept: Symbols are similar if they occur in identical contexts. –Context refers to the symbols that temporally precede and follow the symbol of interest. CBS finds the word “great” and the phrase “very large” to be similar because they are preceded and followed by the same symbols (words)—they have identical contexts

10. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA CBS: Natural Language Processing After reading 25 million words, CBS performed synonym extraction (finding sets of words that occurred in contexts identical to a seed word) No part-of-speech tags were given –In fact, CBS did not do anything that was specific to English, text, or language in general. It would have handled encoder readings, force data, or image properties the same way. Plausible synonym groups were created

11. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA CBS: Perception and concept formation Formulating the concept of the object by compiling specific examples Repeated sequences of 1) a static video background, 2) a dynamic video component created by a moving vehicle, 3) detected motion, and 4) detected sound allow the semantic cluster or concept of to be formed.

12. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA BECCA Operational Diagram When combined, S-Learning and Context-Based Similarity provides BECCA’s full capabilities. –the ability to manage the curse of dimensionality in complex systems –concept generation –hierarchical processing for additional abstraction and sophistication

13. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA Biological Inspiration Discrete motor actions observed in slow movements (Vallbo and Wessberg 1993), infants’ movements (von Hofsten 1991) and repetitive movements (Woodworth 1899) Discrete sensory states proposed by James (1890); evidence seen in carefully structured psychophysical experiments (Gho and Varela 1988, VanRullen and Koch 2003) Periodic processing and storage suggested by thalamcortical loops (Rodriguez, Whitson, and Granger 2004) Sequence storage transition from hippocampus to parahippocampus to cortex described (Eichenbaum et al. 1999) Possible mechanism for classifying and selecting between sequences in cerebellar-cortical-basal- ganglionic loops (Houk 2005)

14. Overview S-Learning Rotary Robot Reaching Grasping Context-Based Similarity Natural Language Processing Perception Bio-Inspiration Applications BECCA Helping robots handle real-world interactions Potential problem spaces for BECCA- driven robots: Learn to manipulate unfamiliar objects Learn complex perceptuo-motor tasks Create high-level abstractions Make predictions about future events Use reasoning to achieve goals Solve poorly-posed problems Generalize experience to novel situations