1. Teaching an Agent by Playing a Multimodal Memory Game: Challenges for Machine Learners and Human Teachers AAAI 2009 Spring Symposium: Agents that Learn from Human Teachers March 23-25, 2009, Stanford University Byoung-Tak Zhang Biointelligence Laboratory School of Computer Science and Engineering Cognitive Science, Brain Science, and Bioinformatics Seoul National University, Seoul 151-744, Korea btzhang@cse.snu.ac.kr http://bi.snu.ac.kr/

2. © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 2 Talk Outline Multimodal Memory Game (MMG) Challenges for Machine Learners Challenges for Human Teachers Toward Self-teaching Cognitive Agents

3. © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ Image Sound Text But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. Image-to-Text Generator (I2T) Image-to-Text Generator (I2T) Text-to-Image Generator (T2I) Text-to-Image Generator (T2I) Text Hint But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. But, I'm getting married tomorrow Well, maybe I am... I keep thinking about you. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. Hint Image Machine Learner Toward Human-Level Machine Learning: Multimodal Memory Game (MMG)

4. Image Sound © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 4 Text Generation Game (from Image) Text I2T Learning by Viewing Learning by Viewing T2I Game Manager Game Manager TextHint T

5. Text Image Sound © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 5 Image Generation Game (from Text) I2T Learning by Viewing Learning by Viewing T2I Game Manager Game Manager HintImage I

6. Characteristics of MMG Game Interactive Multimodal Long-lasting Hard to learn Scalable data Humans as teachers Difficulty controllable Learning by imitation (viewing and watching) © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 6

7. Three Approaches Learning Architecture  Model Learning Strategies  Algorithms Teaching Strategies  Humans © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 7

8. Methods of Machine Learning Symbolic Learning  Version Space Learning  Case-Based Learning Neural (Connectionist) Learning  Multilayer Perceptrons  Self-Organizing Maps  Hopfield Networks Evolutionary Learning  Evolution Strategies  Evolutionary Programming  Genetic Algorithms  Genetic Programming Probabilistic Learning  Bayesian Networks  Boltzmann Machines  Hidden Markov Models  Deep Belief Networks  Hypernetworks Other Machine Learning Methods  Reinforcement Learning  Decision Trees  Boosting Algorithms  Kernel Methods (SVM)  PCA, ICA, LDA etc.

9. Learning with Hypernetworks © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 9 x1 x2 x3 x4 x5 x6 x7 x8x9 x10 x11 x12 x13 x14 x15 [Zhang, DNA12-2006]

10. © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 10 How to Learn from Image-Text Pairs

11. © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 11 How to Generate Image from Text

12. Image-to-Text Translation Results © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 12 AnswerQuery I don't know what happened There's a kitty in my guitar case Maybe there's something I can do to make sure I get pregnant Maybe there's something there's something I … I get pregnant There's a a kitty in … in my guitar case I don't know don't know what know what happened Matching & Completion

13. Text-to-Image Translation Results © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 13 Query Matching & Completion I don't know what happened Take a look at this There's a kitty in my guitar case Maybe there's something I can do to make sure I get pregnant Answer

14. Further Challenges

15. Challenges for Machine Learners Incremental learning Online learning Fast update One-shot learning Predictive learning Memory capacity Selective attention Active learning Context-awareness Persistency Concept drift Multisensory integration © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 15

16. Challenges for Human Teachers Getting feedback Sequencing examples Identifying the weak points Choosing problems Controlling parameters Evaluating progress Estimating difficulty Generating new queries Modeling the effect of learning parameters Catching environmental change Minimal interactions Multimodal interaction © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 16

17. Conclusion Multimodal memory game (MMG)  Highly-interactive lifelong learning scenario  Challenges current machine learning techniques Challenges for machine learners  More attentive, active behavior  Rather than parameter fitting, passive adaptation Human partners  More active role in interacting with the agents The future: Self-teaching cognitive agents  Cognitive learning agents that teach themselves = Active learning agents + cognitively-aware human teachers  Design new queries and test their answers by interacting with humans © 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 17