Presentation is loading. Please wait.

Presentation is loading. Please wait.

Artificial Curiosity Tyler Streeter www.vrac.iastate.edu/~streeter.

Published byCiera Pigott Modified over 9 years ago

Similar presentations

Presentation on theme: "Artificial Curiosity Tyler Streeter www.vrac.iastate.edu/~streeter."— Presentation transcript:

1 Artificial Curiosity Tyler Streeter www.vrac.iastate.edu/~streeter

2 What is Curiosity? An intrinsic drive towards “interesting” situations “Many interesting things are unexpected, but not all unexpected things are interesting.” – Juergen Schmidhuber

3 Curiosity Rewards Humans and many animals learn from reinforcements (rewards/punishment) Examples of primary reinforcements: food, sex, pain… and “interestingness?” Evidence: primate dopamine neurons fire reward signals in novel situations –Kakade, S. & Dayan, P. 2002. Dopamine: Generalization and Bonuses. Neural Networks, 15, 549-559.

4 So What is “Interesting?” “…situations which are neither too predictable nor too unpredictable.” “…the edge of order and chaos in the cognitive dynamics.” –Oudeyer, P. & Kaplan, F. 2004. Intelligent Adaptive Curiosity: A Source of Self-Development. In Proceedings of the 4th International Workshop on Epigenetic Robotics, Volume 117, 127-130. Based on previous experience (very subjective!)

5 Why Artificial Curiosity? Robots and software agents today depend on human teachers; they lack an intrinsic drive to seek new information

6 Hypothesis Curious agents will learn broader, more robust sets of skills (even if they must develop more slowly)

7 Examples of Autonomous Self- Development Stoytchev, A., "Autonomous Learning of Tool Affordances by a Robot", In Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI), Pittsburgh, Pennsylvania, July 9-13, 2005.

8 Examples of Autonomous Self- Development Oudeyer, P., Kaplan, F.; Hafner, V.V., & Whyte, A. 2005. The Playground Experiment: Task-Independent Development of a Curious Robot. In Proceedings of the AAAI Spring Symposium Workshop on Developmental Robotics.

9 How to Model Curiosity Agents must develop predictive models of their environments –“Based on what I’m seeing, what will happen next?” Predictive models enable simulated experiences/imagination

10 How to Model Curiosity Reward agents in situations that become more predictable over time - S chmidhuber, J. 1991. Curious Model-Building Control Systems. In Proceedings of the International Joint Conference on Neural Networks, Singapore, Volume 2, 1458-1463. Prediction Errors Boring (Too Predictable) Time Prediction Errors Boring (Too Unpredictable) Time Prediction Errors Interesting! Time

11 Open Source Implementation Verve: an open source implementation of curious software agents, developed by Tyler Streeter http://verve-agents.sourceforge.net

12 Interactive Demo Tyler Streeter, Curious mobile robot demo, HCI Forum 2006

Download ppt "Artificial Curiosity Tyler Streeter www.vrac.iastate.edu/~streeter."

Similar presentations

Approaches, Tools, and Applications Islam A. El-Shaarawy Shoubra Faculty of Eng.

Approaches, Tools, and Applications Islam A. El-Shaarawy Shoubra Faculty of Eng.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Learning School of Computing, University of Leeds, UK AI23 – 2004/05 – demo 2.

Learning School of Computing, University of Leeds, UK AI23 – 2004/05 – demo 2.
IJCNN 2013 Panel in Higher Emotion 1 Emotion in Brain-Inspired Developmental Networks Juyang (John) Weng Computer Sci., Neurosci., Cognitive Sci. Michigan.

IJCNN 2013 Panel in Higher Emotion 1 Emotion in Brain-Inspired Developmental Networks Juyang (John) Weng Computer Sci., Neurosci., Cognitive Sci. Michigan.
Finding aesthetic pleasure on the edge of chaos: A proposal for robotic creativity Ron Chrisley COGS Department of Informatics University of Sussex Workshop.

Finding aesthetic pleasure on the edge of chaos: A proposal for robotic creativity Ron Chrisley COGS Department of Informatics University of Sussex Workshop.
Reinforcement Learning and the Reward Engineering Principle Daniel Dewey AAAI Spring.

Reinforcement Learning and the Reward Engineering Principle Daniel Dewey AAAI Spring.
Developmental Artificial Intelligence 1 er April 2014 t oliviergeorgeon.com1/31.

Developmental Artificial Intelligence 1 er April t oliviergeorgeon.com1/31.
1.Data categorization 2.Information 3.Knowledge 4.Wisdom 5.Social understanding Which of the following requires a firm to expend resources to organize.

1.Data categorization 2.Information 3.Knowledge 4.Wisdom 5.Social understanding Which of the following requires a firm to expend resources to organize.
ROBOT BEHAVIOUR CONTROL SUCCESSFUL TRIAL OF MARKERLESS MOTION CAPTURE TECHNOLOGY Student E.E. Shelomentsev Group 8Е00 Scientific supervisor Т.V. Alexandrova.

ROBOT BEHAVIOUR CONTROL SUCCESSFUL TRIAL OF MARKERLESS MOTION CAPTURE TECHNOLOGY Student E.E. Shelomentsev Group 8Е00 Scientific supervisor Т.V. Alexandrova.
Chapter 10 Artificial Intelligence © 2007 Pearson Addison-Wesley. All rights reserved.

Chapter 10 Artificial Intelligence © 2007 Pearson Addison-Wesley. All rights reserved.
Intrinsically Motivated Hierarchical Reinforcement Learning Andrew Barto Autonomous Learning Lab Department of Computer Science University of Massachusetts.

Intrinsically Motivated Hierarchical Reinforcement Learning Andrew Barto Autonomous Learning Lab Department of Computer Science University of Massachusetts.
Dopamine, Uncertainty and TD Learning CNS 2004 Yael Niv Michael Duff Peter Dayan Gatsby Computational Neuroscience Unit, UCL.

Dopamine, Uncertainty and TD Learning CNS 2004 Yael Niv Michael Duff Peter Dayan Gatsby Computational Neuroscience Unit, UCL.
Introduction to Neural Network Justin Jansen December 9 th 2002.

Introduction to Neural Network Justin Jansen December 9 th 2002.
Cooperative Q-Learning Lars Blackmore and Steve Block Expertness Based Cooperative Q-learning Ahmadabadi, M.N.; Asadpour, M IEEE Transactions on Systems,

Cooperative Q-Learning Lars Blackmore and Steve Block Expertness Based Cooperative Q-learning Ahmadabadi, M.N.; Asadpour, M IEEE Transactions on Systems,
Mental Development and Representation Building through Motivated Learning Janusz A. Starzyk, Ohio University, USA, Pawel Raif, Silesian University of Technology,

Mental Development and Representation Building through Motivated Learning Janusz A. Starzyk, Ohio University, USA, Pawel Raif, Silesian University of Technology,
Intelligent Agents revisited.

Intelligent Agents revisited.
IJCNN, International Joint Conference on Neural Networks, San Jose 2011 Pawel Raif Silesian University of Technology, Poland, Janusz A. Starzyk Ohio University,

IJCNN, International Joint Conference on Neural Networks, San Jose 2011 Pawel Raif Silesian University of Technology, Poland, Janusz A. Starzyk Ohio University,
A.M. Florea, Cognitive systems, COST Action IC0801 – WG1, 15 December, Ayia Napa, Cyprus.

A.M. Florea, Cognitive systems, COST Action IC0801 – WG1, 15 December, Ayia Napa, Cyprus.
Curious Reconfigurable Robots Kathryn Merrick ITEE Seminar – Friday 29 th August Collaborators: Dayne Schmidt, Elanor Huntington.

Curious Reconfigurable Robots Kathryn Merrick ITEE Seminar – Friday 29 th August Collaborators: Dayne Schmidt, Elanor Huntington.
Fourth International Symposium on Neural Networks (ISNN) June 3-7, 2007, Nanjing, China Online Dynamic Value System for Machine Learning Haibo He, Stevens.

Fourth International Symposium on Neural Networks (ISNN) June 3-7, 2007, Nanjing, China Online Dynamic Value System for Machine Learning Haibo He, Stevens.

Similar presentations

Ads by Google