1. The Role of Prior Knowledge in Human Reconstructive Memory Mark Steyvers Pernille Hemmer University of California, Irvine 1

2. Interaction between Prior Knowledge & Episodic Memory 2 Episodic Memory Prior Knowledge, Semantic Memory, “Schemas” “What objects do you remember from your hotel room?” “What coffee did you order from Starbucks last week?” reconstructive memory

3. Research questions Machine learning How can we formalize prior knowledge (“schemas”)? Psychology How do humans integrate memory and prior knowledge? What errors do people make? Interface between human & machine learning Can we explain human error with rational inference principles? Can we build better information retrieval systems by taking cognitive processes into account? 3

4. Remembering Objects from a Graduate Office Brewer & Treyens (1981) chair desk skull books (30% of subjects)

5. Experiments perception: list objects you see in a scene memory: list objects you remember from a scene guessing: list objects you might see in a kitchen/office/… 5 Dining Hotel Kitchen

6. Human Data: Precision vs. Output Position 6 guessing 10 Secs. 2 Secs Precision

7. Bayesian Analysis of Reconstructive Memory 7 Posterior What objects were studied given noisy memory contents? Prior How likely are these objects a priori? Likelihood How likely is this memory content given these objects were studied?

8. Problem 1: Infer prior knowledge 8 kitchen1 kitchen3kitchen2 Prior Knowledge ? Hierarchical Beta processes (Thibaux & Jordan, 2007); related to Indian Buffet process

9. Hierarchical Beta Process (Thibaux & Jordan, 2007) 9 “ Kitchen ” Kitchen65 objects participants Kitchen45 General knowledge Object probs. for specific image Reported objects y

10. Problem 2: Recall using Prior + Episodic M. 10 ? Prior Knowledge Actual objects in image Noisy Memory Content semantic memory episodic memory

11. 11 Data Model prior only

12. Limitations of Model Independence assumption between objects Recall often clusters correlated objects “computer”  “mouse”  “keyboard” 12

13. Reconstructing the Style of Written Digits 13 Study Reconstructed studied digit reconstructed personal prior Digit space from “Earth Mover” distances

14. Reconstructing Drawings from Memory 14 Studied Drawing Reconstruction

15. Reconstructing Lists of Words 15 Study this list: PEAS, CARROTS, BEANS, SPINACH, LETTUCE, HAMMER, TOMATOES, CORN, CABBAGE, SQUASH HAMMER, PEAS, CARROTS,...

16. Conclusion Prior knowledge has strong effect on memory Guessing based on prior knowledge leads to quite good performance Machine learning methods help formalize “schemas” Useful to understand human memory when designing information retrieval systems 16

17. Thanks! 17