1. EE141 EE690 Design of Embodied Intelligence Janusz Starzyk EECS, Ohio University

2. EE141  “…Perhaps the last frontier of science – its ultimate challenge- is to understand the biological basis of consciousness and the mental process by which we perceive, act, learn and remember..” from Principles of Neural Science by E. R. Kandel et al. E. R. Kandel won Nobel Price in 2000 for his work on physiological basis of memory storage in neurons. His family roots are in Kolomyje – so he once told “as with all bright people, my roots are in Poland”  “… The question of intelligence is the last great terrestrial frontier of science...” from Jeff Hawkins On Intelligence. Jeff Hawkins founded the Redwood Neuroscience Institute devoted to brain research. He co-founded Palm Computing and Handspring Inc. Intelligence AI’s holy grail From Pattie Maes MIT Media Lab

3. EE141  Traditional Artificial Intelligence  Embodied Intelligence (EI)  Challenges of EI  We need to know how  We need means to implement it  We need resources to build and sustain its operation  Promises of EI  To economy  To society Outline

4. EE141 Intelligence From http://www.indiana.edu/~intell/map.shtml Mainstream Science on Intelligence December 13, 1994: An Editorial With 52 Signatories, History, and Bibliography by Linda S. Gottfredson, University of Delaware Intelligence is a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience.

5. EE141 Various Definitions of Intelligence  The American Heritage Dictionary: l The capacity to acquire and apply knowledge. l The faculty of thought and reason.  Webster Dictionary: l The act or state of knowing; the exercise of the understanding. l The capacity to know or understand; readiness of comprehension;  Wikipedia – The Free Encyclopedia: l The capacity to reason, plan, solve problems, think abstractly, comprehend ideas and language, and learn.reasonplansolve problemsabstractlylanguagelearn  The classical behavioral/biologists: l The ability to adapt to new conditions and to successfully cope with life situations.  Dr. C. George Boeree, professor in the Psychology Department at Shippensburg University:Psychology Department Shippensburg University l A person's capacity to (1) acquire knowledge (i.e. learn and understand), (2) apply knowledge (solve problems), and (3) engage in abstract reasoning.  Stanford University Professor of Computer Science Dr. John McCarthy, a pioneer in AI: l The computational part of the ability to achieve goals in the world.

6. EE141 Traditional AI Embodied Intelligence  Abstract intelligence  attempt to simulate “highest” human faculties: –language, discursive reason, mathematics, abstract problem solving  Environment model  Condition for problem solving in abstract way  “brain in a vat”  Embodiment  knowledge is implicit in the fact that we have a body –embodiment is a foundation for brain development  Intelligence develops through interaction with environment  Situated in a specific environment  Environment is its best model

7. EE141 Design principles of intelligent systems from Rolf Pfeifer “Understanding of Intelligence”, 1999  Interaction with complex environment  cheap design  ecological balance  redundancy principle  asynchronous  parallel, loosely coupled processes  sensory-motor coordination  value principle Agent Drawing by Ciarán O’Leary- Dublin Institute of Technology

8. EE141 Embodied Intelligence  Definition  Embodied Intelligence (EI) is a mechanism that learns how to survive in a hostile environment –Mechanism: biological, mechanical or virtual agent with embodied sensors and actuators –EI acts on environment and perceives its actions –Environment hostility is persistent and stimulates EI to act –Hostility: direct aggression, pain, anxiety or scarcity of resources –EI learns so it must have associative self-organizing memory –Knowledge is acquired by EI

9. EE141 Embodiment of Mind Definition  Embodiment of a mind is a mechanism under the control of the intelligence core that contains sensors and actuators connected to the core through communication channels. Environment Intelligence core Embodiment Sensors Actuators

10. EE141 Embodiment of Mind  Necessary for development of intelligence  Hosts brain’s interfaces that interact with environment  Not necessarily constant or in the form of a physical body  Boundary transforms modifying brain’s self-determination Environment Intelligence core Embodiment Sensors Actuators

11. EE141  Brain learns own body’s dynamic  Self-awareness results from identification with own embodiment  Embodiment can be extended by using tools and machines  Successful operation depends on correct perception of environment and own embodiment Embodiment of Mind

12. EE141 Broca’s area Pars opercularis Motor cortexSomatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex Brain Organization While we learn its functions can we emulate its operation?

13. EE141 How can we design intelligence?  We need to know how  We need means to implement it  We need resources to build and sustain its operation

14. EE141 Requirements for Embodied Intelligence  State oriented  Learns spatio-temporal patterns  Situated in time and space  Learning  Perpetual learning  Screening for novelty  Value driven  Goal creation  Competing goals  Emergence  Artificial evolution  Self-organization

15. EE141 INPUTOUTPUT Simulation or Real-World System Task Environment Agent Architecture Long-term Memory Short-term Memory Reason Act Perceive RETRIEVALLEARNING EI Interaction with Environment From Randolph M. Jones, P : www.soartech.com

16. EE141 Challenges of Embodied Intelligence  Development of sensory interfaces  Active vision  Speech processing  Tactile, smell, taste, temperature, pressure sensing  Additional sensing –Infrared, radar, lidar, ultrasound, GPS, etc. –Can too many senses be less useful?  Development of reinforcement interfaces  Energy, temperature, pressure, acceleration level  Teacher input  Development of motor interfaces  Arms, legs, fingers, eye movement

17. EE141 Challenges of Embodied Intelligence (cont.)  Finding algorithmic solutions for  Association, memory, sequence learning, invariance building, representation, anticipation, value learning (pain reduction), goal creation, planning  Development of circuits for neural computing  Determine organization of artificial minicolumn  Self-organized hierarchy of minicolumns for sensing and motor control  Self-organization of goal creation pathway

18. EE141  V. Mountcastle argues that all regions of the brain perform the same computational algorithm V. Mountcastle  Groups of neurons (minicolumns) have the same structure and are connected in a pseudorandom way  Minicolumns organized in macrocolumns VB Mountcastle (2003). Introduction [to a special issue of Cerebral Cortex on columns]. Cerebral Cortex, 13, 2-4. Human Intelligence - Uniform Structure

19. EE141 Selective Processing in Minicolumns  Sensory inputs are represented by gradually more abstract features in the sensory hierarchy  Use “sameness principle” of the observed objects to detect and learn feature invariance  Learn to store temporal sequences  Use random wiring to preselect sensory features  Use feedback for input prediction and screen input information for novelty  Use redundant structures of sparsely connected processing elements

20. EE141 Interactions between Minicolumns Positive Reinforcement Negative Reinforcement Sensory Inputs Motor Outputs Value center Sensory processing Motor processing Pleasure Pain Learn Control Associate Direct

21. EE141  Sensory neurons are responsible for representing environment  receive inputs from sensors or sensory neurons on lower level  represent the environment  receive feedback input from motor and higher level sensory neurons  help to activate motor and reinforcement neurons  Motor neurons are responsible for actions and skills  are activated by reinforcement and sensory neurons  activate actuators or provide an input to lower level motor neurons  provide planning inputs to sensory neurons  Reinforcement neurons are responsible for building the value system, goal creation, learning, and exploration  receive inputs from reinforcement neurons on the lower level  receive inputs from sensory neurons  provide inputs to motor neurons  initiate learning and force exploration Functions of Neurons in Minicolumns

22. EE141 Selective Sensory-Motor Pathways  Hierarchical sensory and motor pathways  Branching off from more general to more specific  Easy to expand to higher levels  Gradual loss of interconnect plasticity towards inputs inputs internal representations

23. EE141 Environment ………… …... …… Sensory neurons in a minicolumn Selective Sensory-Motor Pathways Increasing connection’s plasticity 10 6 neurons 10 11 neurons 10 neurons Sensory pathway 10 neurons 10 4 neurons Activation pathway

24. EE141 General characteristics: Hierarchical structure Minicolumn processing Spatial and temporal association via interconnections and dual neurons Feedback connections Selective adaptation Function: Invariant representation Anticipation Screening for novelty Value learning Organization of Sensory-motor Pathways

25. EE141 Sensory-motor Coordination R: representation E: expectation A: association D: direction P: planning

26. EE141 Goal Driven Behavior  Goal driven behavior is one of the required elements of intelligence  Perceptions and actions are activated selectively to serve the machine’s objectives  In the existing EI models, the goal is defined by designers and is given to the learning agent  Humans and animals create their own goals  The goal creation may be one of the most important elements of EI mechanism

27. EE141 Goal Creation  Goal creation ability is essential for developing intelligence  We will create goals based on simple structures interacting with sensory and motor pathways  Goals must be built and understood in a similar way to building perceptions  More complex goals can be understood only if representations for these goals are build  Goal creation should result from EI interaction with environment, by perceiving successes or failures of its actions

28. EE141 Pain-center and Goal Creation  Simple Mechanism  Creates hierarchy of values  Leads to formulation of complex goals  Reinforcement neurotransmitter: Pain increase Pain decrease  Forces exploration + - Environment Sensor Motor Pain level Dual pain level Pain increase Pain decrease (-) (+) Excitation (-) (+)

29. EE141 Abstract Goal Creation  The goal is to reduce the primitive pain level  Abstract goals are created to satisfy the primitive goals Expectation Association Inhibition Reinforcement Connection Planning -+ PainDual pain Food refrigerator -+ Stomach Abstract pain (Delayed memory of pain) “food”becomes a sensory input to abstract pain center Sensory pathway (perception, sense) Motor pathway (action, reaction) Primitive Level Level I Level II Eat Open

30. EE141 The Three Pathways Combined  Goal creation, sensory and motor pathways interact on different hierarchy levels  Pain driven goal creation sets goal priorities  The tree pathways emerge together from interaction with the environment Pain tree I Pain tree II Motor pathway Sensory pathway Pain center to motor Sensor to motor Sensor to pain center

31. EE141 How can we design intelligence?  We need to know how  We need means to implement it  Hardware self-organization  Sparsity of interconnect  Functionality requirements  EI Design Efforts  We need resources to build and sustain its operation

32. EE141 We need means to implement it  What is needed is a self-organizing hardware  Saves design work  Increases hardware reliability  Improves production yield  Lowers design cost  Basic mechanism for self-organized hardware is pruning of random interconnections  Nature does it  Activity triggered “growth” of new neurons  Spare neurons used in the later stages of learning

33. EE141  Brain is self-organizing and sparse Human Brain at Birth 6 Years Old 14 Years Old Rethinking the Brain, Families and Work Institute, Rima Shore, 1997. We need means to implement it

34. EE141 Thompson, R. A., & Nelson, C. A. (2001). Developmental science and the media: Early brain development. American Psychologist, 56(1), 5-15. Synaptic Density over the Lifespan

35. EE141 Sparse Connectivity The brain is sparsely connected. A neuron in cortex may have on the order of 100,000 synapses. There are more than 10 10 neurons in the brain. Fractional connectivity is very low: 0.001%. Implications:  Connections are expensive biologically.  Short local connections are cheaper than long ones.

36. EE141  Feature extraction – using WTA  Generalization – using overlapping regions Hierarchical Self-organizing Learning Memory (HSOLM)

37. EE141 Hierarchical Self-organizing Learning Memory  3-layer R-net in hierarchical structure  Reduced connection density  The winner pathway replaces the WTA  Established using feed-back and local competition number of output links from the lower level nodes

38. EE141 Self-Organizing Learning Arrays SOLAR

39. EE141 Wiring in SOLAR Initial wiring and final wiring selection for credit card approval problem

40. EE141 Associative SOLAR

41. EE141 Sequence Learning Mechanism

42. EE141 Software or hardware?  Sequential  Error prone  Require programming  Low cost  Well developed programming methods  Concurrent  Robust  Require design  Significant cost  Hardware prototypes hard to build SoftwareHardware

43. EE141 Future software/hardware capabilities Human brain complexity 100 FPGAs 10 FPGAs

44. EE141 How can we design intelligence?  We need to know how  We need means to implement it  We need resources to build and sustain its operation

45. EE141 Major Intelligent Systems Design Efforts  GOMS - Goals, Operators, Methods, and Selection Rules  Proposed by Card, Moran, and Newell (1983), for modeling and describing human task performance  SOAR - State, Operator, Application, Result  Based on the book by Newell, A. (1990), Unified Theories of Cognition.  ACT-R (Adaptive Control of Thought - Rational )  Described by Anderson, J. (1993), Rules of the Mind  Artificial General Intelligence – Adaptive A.I. Inc.  http://adaptiveai.com/  Hierarchical Temporal Memory – Numenta Inc. Irvine Sensors Corporation (Costa Mesa, CA) 3DANN “Brain On Silicon” will not be just a science fiction!  http://www.numenta.com/

46. EE141  GOMS - Goals, Operators, Methods, and Selection Rules  Inspired by psychology of immediate action  Explicit hierarchical task decompositions  Explicit pairing of goals with tasks  Belief, Goal and Plan representation and selection  Primary Engineering Activities  Cognitive task analysis  Design of goal hierarchy  Definition of plans and selection rules  Definition of primitive operators and resource constraints EI Design Efforts Proposed by Card, Moran, and Newell (1983), for modeling and describing human task performance From slide by Dan Glaser

47. EE141  SOAR - State, Operator, Application, Result  Problem space and physical symbols  Minimal set of rules to support intelligent behavior  Belief, Goal and Plan representation and selection  Primary Engineering Activities  Cognitive task analysis  Identification of primitive operations  Design of belief and goal hierarchies  Design of situation interpretation hierarchies  Interactions between knowledge units  http://sitemaker.umich.edu/soar Newell, A. 1990. Unified Theories of Cognition. Cambridge, Massachusetts: Harvard University Press. EI Design Efforts

48. EE141  ACT-R (Adaptive Control of Thought - Rational ):  a cognitive architecture inspired by psychological models of memory, skills, and learning  Optimization-oriented integrated memory, action, and learning  Procedural knowledge units with explicit retrieval  Uses if then production rules  Belief, Goal and Plan representation and selection  Primary Engineering Activities  Cognitive task analysis  Identification of procedural and declarative knowledge  Design of semantic associations and retrievable goal hierarchies  Identification of primitive operations with logical and associative context Anderson, John. Rules of the Mind, Lawrence Erlbaum Associates, Hillsdale, NJ (1993). EI Design Efforts

49. EE141 EI Design Efforts  Hierarchical Temporal Memory – Numenta Inc.  Uses hierarchy of spatio-temporal associations  Discover causes in the world  Learns complex goal-oriented behavior  Makes predictions while learning  Combines unsupervised and supervised learning to make associations  Aims at exceeding human level cognition  Primary Engineering Activities  Cognitive structure analysis  Pattern learning in hierarchical memories  Temporal sequence learning mechanism  Goal creation  http://www.numenta.com/ Die Stack Irvine Sensors Corporation (Costa Mesa, CA)

50. EE141 EI Design Efforts  Artificial General Intelligence – Adaptive A.I. Inc.  Has general cognitive ability  Acquires knowledge and skills  Uses bidirectional, real-time interaction  Selects information and uses an adaptive attention  Unsupervised and self-supervised learning  Adaptive and self-organizing data structure  Processes dynamic patterns  Primary Engineering Activities  Explicitly engineering functionality  Conceptual design  General proof of concept  Animal level cognition  http://adaptiveai.com/ Irvine Sensors Corporation (Costa Mesa, CA) 3DANN “Brain On Silicon” will not be just a science fiction!

51. EE141 How can we design intelligence?  We need to know how  We need means to implement it  We need resources to build and sustain its operation

52. EE141 From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006 Resources – Evolution of Electronics

53. EE141 By Gordon E. Moore

54. EE141

55. From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006 Clock Speed (doubles every 2.7 years)

56. EE141 Doubling (or Halving) times  Dynamic RAM Memory “Half Pitch” Feature Size5.4 years  Dynamic RAM Memory (bits per dollar)1.5 years  Average Transistor Price1.6 years  Microprocessor Cost per Transistor Cycle1.1 years  Total Bits Shipped1.1 years  Processor Performance in MIPS1.8 years  Transistors in Intel Microprocessors2.0 years  Microprocessor Clock Speed2.7 years From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

57. EE141 Why should we care? Source: SEMATECH

58. EE141 Percent of die area that must be occupied by memory to maintain SOC design productivity Design Productivity Gap  Low-Value Designs? Source = Japanese system-LSI industry

59. EE141 2002201020202030 Biomimetics and Bio-inspired Systems Impact on Space Transportation, Space Science and Earth Science Mission Complexity Biological Mimicking Embryonics Extremophiles DNA Computing Brain-like computing Self Assembled Array Artificial nanopore high resolution Mars in situ life detector Sensor Web Biological nanopore low resolution Skin and Bone Self healing structure and thermal protection systems Biologically inspired aero-space systems Space Transportation

60. EE141 NanoEngineer-1 is an Open Source (GPL) project sponsored by Nanorex, Inc. It is an interactive 3D nanomechanical engineering program From Nanorex Inc. Revolutions in electronics and computing will allow reconfigurable, autonomous, "thinking" spacecraft http://ipt.arc.nasa.gov/nanotechnology.html Planetary Gear Universal Joint Nanosystems

61. EE141 From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

62. EE141 Promises of embodied intelligence  To society  Advanced use of technology –Robots –Tutors –Intelligent gadgets  Society of minds –Superhuman intelligence –Progress in science –Solution to societies’ ills  To industry  Technological development  New markets  Economical growth ISAC, a Two-Armed Humanoid Robot Vanderbilt University

63. EE141 Transhumanist values  Nothing wrong about “tampering with nature”  Individual choice in the use of enhancement technologies;  Peace, international cooperation, anti-proliferation of WMDs  Improving understanding (research and public debate; critical thinking; open-mindedness; scientific progress; open discussion of the future)  Getting smarter (individually; collectively; develop machine intelligence)  Pragmatism; engineering and entrepreneur-spirit; “can-do” attitude  Diversity (species, race, religious creed, sexual orientation, life style, etc.)  Saving lives (life extension, anti-aging research, and cryonics) Copyright Institute for Ethics and Emerging Technologies 2004-2005

64. EE141 Sounds like science fiction  If you’re trying to look far ahead, and what you see seems like science fiction, it might be wrong.  But if it doesn’t seem like science fiction, it’s definitely wrong. From presentation by Feresight Institute

65. EE141

66. Embodied Artificial Intelligence Based on: [1] E. R. Kandel et al. Principles of Neural Science, McGraw-Hill/Appleton & Lange; 4 edition, 2000. [2] F. Inda, R. Pfeifer, L. Steels, Y. Kuniyoshi, “Embodied Artificial Intelligence,” International seminar, Germany, July 2003. [3] R. Chrisley, “Embodied artificial intelligence, ” Artificial Intelligence, vol. 149, pp.131-150, 2003. [4] R. Pfeifer and C. Scheier, Understanding Intelligence, MIT Press, Cambridge, MA, 1999. [5] R. A. Brooks, “Intelligence without reason,” In Proc. IJCAI-91. (1991) 569-595. [6] R. A. Brooks, Flesh and Machines: How Robots Will Change Us, (Pantheon, 2002). [7] R. Kurzweil The Age of Spiritual Machines: When Computers Exceed Human Intelligence, (Penguin, 2000).