1. When the Shoe Fits: Cross-Situational Learning in Realistic Learning Environments Tamara N. Medina 1 John Trueswell 1 Jesse Snedeker 2 Lila Gleitman 1 1 Institute for Research in Cognitive Science, University of Pennsylvania 2 Department of Psychology, Harvard University

3. 1998, Third Edition, Completely Revised Where are your shoes? Oh, what dirty shoes! Lets take your shoes off. Ill put your shoes on. Look what nice new shoes. A baby hears a word like shoes, for example, over and over again in daily life as the one constant sound in a large variety of statements. In one day you may say to him: That word shoes is the one sound which occurs in all those sentences and it is always associated with those things that go on his feet. Eventually he will associate the spoken sound with the objects and when he has made that association, he will have learned what the word shoes means.

4. Cross-Situational Learning Find a set of possible meanings in each situation and intersect those sets across all situations in which a word occurs to determine the meaning for that word. Siskind, J.M. (1996, Cognition) Its not so easy! –Augustine, Locke, Quine, Gleitman, Fodor, Siskind, etc. Frame / Level of Description Animal? Dog? Terrier? Fido? Friendly? Referential Uncertainty Which referent?

5. Frame Problem Solved? Xu & Tenenbaum (2007): learn appropriate extensions of a word via Bayesian inference (note suspicious coincidences) VASH

6. Reference Problem Solved? Yu & Smith (2007): learn word-object associations in spite of referential uncertainty DOON … VASH … MIPEN … ZANT

7. VASH ??

8. VASH ! Goal: Explore cross-situational word learning using naturalistic settings: both the cluttered and potentially uninformative or misleading environments and these somewhat more transparent ones.

9. Overview Adaptation of the Human Simulation Paradigm (Gillette et al., 1999) Norming Study Current Study 2 measures to evaluate word learning

10. Adaptation of Human Simulation Paradigm ( Gillette et al., 1999 ) Selected stimuli based on results of earlier norming study: Gertner, Y., Fisher, C., Gleitman, G., Joshi, A., & Snedeker, J. (In progress). Machine implementation of a verb learning algorithm. Large video corpus of parent-child interactions in natural settings (home, outdoors, etc.): Snedeker, J. (2001). Interactions between infants (12-15 months) and their parents in four settings. Unpublished corpus.

11. Norming Study Identified 48 most frequently occurring content words. Randomly selected six instances of each word. Each instance was edited into a 40-second vignette. Sound turned off. –Visual context only cue to word meaning, placing viewers in the situation of the early word learner. Utterance of target word indicated by a BEEP. Gertner, Y., Fisher, C., Gleitman, G., Joshi, A., & Snedeker, J. (In progress). Machine implementation of a verb learning algorithm.

12. 30 sec 10 sec (silence) Drawings courtesy of Emily Trueswell

13. Subject Guesses (Target Word = Shoe) Subject Guesses (Target Word = Horse) … Subject Guesses (Target Word = Shoe) No opportunity for cross-situational learning in norming study 8% correct83% correct0% correct High Informative (>50% correct) Low Informative (<33% correct) 7% of Vignettes = High Informative (atypical) 90% of Vignettes = Low Informative (typical) Gertner, Y., Fisher, C., Gleitman, G., Joshi, A., & Snedeker, J. (In progress). Machine implementation of a verb learning algorithm.

14. Questions Does the observation of multiple naturalistic learning instances generate a gradually increasing learning curve? With regard to informativeness, given only the Low Informative vignettes, is cross-situational learning successful? Or is a High Informative instance necessary? If learners are building an interpretation across instances, does it matter when the High Informative instance occurs?

15. Current Study Similar to norming study, BUT allows for cross-situational learning

16. 30 sec 10 sec (silence) Current Study Allows for Cross-Situational Learning VASH

17. (Target Word = Shoe) Subject makes guess Subject rates Confidence (1 to 5) … Opportunity for cross-situational learning MIPEN (Target Word = Horse) Subject makes guess Subject rates Confidence (1 to 5) VASH (Target Word = Shoe) Subject makes guess Subject rates Confidence (1 to 5) Final Conjectures and Confidence Ratings for each word

18. Manipulated the distribution of informative events For each of 8 Target nouns, there were: –1 High Informative vignette (>50% of participants correct in norming study) –4 Low Informative vignettes (<33% of participants correct in norming study) 4 Filler nouns –5 Low Informative vignettes Participants assigned to one of 4 orders: –High Informative First:H-L-L-L-L –High Informative Middle: L-L-H-L-L –High Informative Last:L-L-L-L-H –High Informative Absent: L-L-L-L-L (fifth vignette is a repeat of the first)

19. Accuracy Across Vignettes

24. Interim Summary: What have we learned about learning? 1.Gradual learning from partially informative instances is small to nonexistent. 2.Successful learning depends on the presence of a High Informative instance. (Epiphany!) 3.Low Informative instances have a corrupting influence on later-occurring High Informative instances. (Cross-situational learning of the bad sort.)

25. Epiphany! Successful learning depends on the presence of a High Informative instance. Explicit and immediate insight? After using evidence from later instances? –High Informative instance provides key for interpreting later instances.

26. Confidence on Correct Guesses across Vignettes

30. Implications Shape of the word learning curve may be very different than what cross-situational learning models (e.g., Yu & Smith, 2007) have suggested: Rapid Incremental

31. Implications Successful word learning from cross-situational observation requires the occurrence of a highly informative instance. –But must it occur first? Greater delay between instances of a novel word: Every day is a new day. Multiple High Informative learning instances. –Previous studies which show striking rapid word learning are such cases. Less weight on interpretations of Low Informative instances. Logically, no!

32. Implications A High Informative instance is the first step in successful cross-situational word learning. –Prior Low Informative instances might not be remembered over time. –Later Low Informative instances become useful (confirmatory evidence?) Supported by rising confidence levels after a High Informative vignette.

33. Cross-situational learning does work, but only when the shoe fits.

34. Accuracy Confidence

35. Perseverance of First Guess