1. Statistical Learning in Infants (and bigger folks)

2. Statistical Learning Neural network models emphasize the value of statistical information in language –What information can be extracted from this? –Is this sufficient to account for human performance? –Are humans able to perform this kind of analysis? –If so, does it contribute to an understanding of the uniquely human ability to learn language?

3. Saffran, Aslin, & Newport (1996) 8-month old infants –Passive exposure to continuous speech (2 mins) bidakupadotigolabubidaku… –Test (Experiment #2) bidakubidakubidakubidakubidaku… kupadokupadokupadokupadokupado… –Infants listen longer to unfamiliar sequences –Transitional Probabilities bi da ku pa do ti 1.0.33 Jenny Saffran Dick Aslin Elissa Newport

7. What is it good for? Word Learning –Transitional probabilities: local minima = word boundaries –Saffran’s example: ‘pretty baby’ /prItibebi/ p (ti|prI) = 0.8 p (be|ti) = 0.03 How else could children segment words? –Words in isolation (Peters, 1983; Pinker, 1984) –Stress-based segmentation: 90% of English words are stress-initial (Cutler & Carter, 1987) –Phonotactic segmentation, e.g., *dnight (Gambell & Yang, 2005)

8. Are Local Minima Effective? Gambell & Yang (2005) - –Adult input to children from 3 corpora in CHILDES –226,178 words, 263,660 syllables –Precision: hits/(hits + false alarms)41.6% Recall: hits/(hits + misses)23.3%

9. More Statistical Learning Additional Stimulus types –Tones –Shapes –etc. Additional species…

10. Cotton-top Tamarin “Jackendoff” Marc Hauser

15. Where do constraints come from? Substantive Constraints If the statistical learning mechanism is able to pick up regularities that go beyond those found in natural languages, then there must be additional substantive linguistic constraints that provide the restrictions on natural languages Constraints on Learning & Processing “… some of the constraints on natural language structure might arise from constraints on the computational abilities this mechanism exhibits.” (p. 130)

21. Albert Bregman

22. k t b | | | C - V - C - V - C | | a a Autosegmental Phonology

23. Where do constraints come from? “ This compatibility between learning and languages in turn suggests that natural language structures may be formed, at least in part, by the constraints and selectivities of what human learners find easy to acquire. ” (p. 159)

24. Where do constraints come from? How well does this generalize?

25. Where do constraints come from? Substantive Constraints vs. Constraints on Learning or Processing Rather than removing the need for substantive constraints, Newport ’ s approach seems to shift the burden of explanation onto the theory of representations

30. Jenny Saffran Curr. Dir. Psych. Sci., 12: 110-114 (2003)

32. Experiment 1 - Syllable Size Step 1: Pattern Induction –Regime A: CVCV words, e.g., boga, diku –Regime B: CVCCVC words, e.g., bikrub, gadkug Step 2: Segmentation –4 words: [baku, dola], [tupgod, girbup] –Continuous stream: tupgodbakugirbupdolabaku… Step 3: Testing –Same words used in segmentation: [baku, dola], [tupgod, girbup] –Infants listened longer to words consistent w/ induced pattern

33. Experiment 2 - Phonotactics Step 1: Pattern Induction –Regime A: -V+V syllables, e.g., todkad, pibtug –Regime B: +V-V syllables, e.g., dakdot, gutbip Step 2: Segmentation –4 words: [kibpug, pagkob], [bupgok, gikbap] –Continuous stream: pagkobbupgokgikbapkibpug… Step 3: Testing –Same words used in segmentation: [kibpug, pagkob], [bupgok, gikbap] –Infants listened longer to words consistent w/ induced pattern

34. Experiment 3 - Unnatural Phonotactics Experiment 2 –-V+V pattern is stated over a feature-based class: /p,t,k/ vs. /b,d,g/ Experiment 3 –Modify segment ‘groupings’: /p,d,k/ vs. /b,t,g/ –Other details just like Experiment 2 –No listening preference at test phase

35. Conclusion “To the extent that patterns that do not occur in natural languages are more difficult to acquire, we may consider the possibility that constraints on how infants learn may have served to shape the phonology of natural languages. Patterns that are difficult to acquire are less likely to persist cross-linguistically than those that are easily learned. Thus, languages may exploit devices such as voicing regularities in part because they are readily acquired by young learners.” [Saffran & Thiessen 2003, p. 491]

36. Marcus et al. (1999) Training –ABA:ga na gali ti li –ABB:ga na nali ti ti Testing –ABA:wo fe wo –ABB:wo fe fe Gary Marcus

37. #1: ABB vs. ABA #2: ABB vs. ABA #3: ABB vs. AAB

39. (Pena et al. 2002)

42. Rule learning in infants is domain-specific Marcus, Fernandes & Johnson, submitted

49. So what are we learning…?

51. Verb Argument Structure

52. Baker (1979) Alternating Verbs –John gave a cookie to the boy. John gave the boy a cookie. –Mary showed some photos to her family. Mary showed her family some photos. Non-Alternating Verbs –John donated a painting to the museum. *John donated the museum a painting. –Mary displayed her art collection to the visitors. *Mary displayed the visitors her art collection Learnability problem: how to avoid overgeneralization

53. Verb Argument Structure “Locative Verbs” Sally poured the water into the glass. *Sally poured the glass with water. *Sally filled the water into the glass. Sally filled the glass with water. Sally piled the books on the table. Sally piled the table with books.

54. Verb Argument Structure “Locative Verbs” Sally poured the water into the glass. *Sally poured the glass with water. *Sally filled the water into the glass. Sally filled the glass with water. Sally piled the books on the table. Sally piled the table with books. Figure-verbs -- manner of motion pour, spill, drip, shake, etc. Ground-verbs -- change of state fill, cover, decorate, soak, etc. Alternator-verbs -- manner & change pile, scatter, load, etc.

55. Verb Classes Assumptions 1. Linking rules are consistent across languages 2. Linking rules need not be learned

56. Seidenberg (1997, Science) Locative Verb Constructions –John poured the water into the cup *John poured the cup with water –*Sue filled the water into the glass Sue filled the glass with water –Bill loaded the apples onto the truck Bill loaded the truck with apples “Connectionist networks are well suited to capturing systems with this character. Importantly, a network configured as a device that learns to perform a task such as mapping from sound to meaning will act as a discovery procedure, determining which kinds of information are relevant. Evidence that such models can encode precisely the right combinations of probabilistic constraints is provided by Allen (42), who implemented a network that learns about verbs and their argument structures from naturalistic input.” (p. 1602)

57. Seidenberg (Science, 3/14/97) “Research on language has arrived at a particularly interesting point, however, because of important developments outside of the linguistic mainstream that are converging on a different view of the nature of language. These developments represent an important turn of events in the history of ideas about language.” (p. 1599)

58. Seidenberg (Science, 3/14/97) “A second implication concerns the relevance of poverty-of-the-stimulus arguments to other aspects of language. Verbs and their argument structures are important, but they are language specific rather than universal properties of languages and so must be learned from experience.” (p. 1602)

59. Verb Argument Structure Distributional learning of linking rules (Seidenberg 1997, Science) a. Innate knowledge of nature of solution b. Model needs to select relevant semantic features from a pool of candidates “Verbs and their argument structures are important, but they are language specific rather than universal properties of languages and so must be learned from experience.” (p. 1602) (Allen & Seidenberg, 1997)

60. Allen’s Model Learns associations between (i) specific verbs & argument structures and (ii) semantic representations Feature encoding for verbs, 360 features [eat]: +act, +cause, +consume, etc. [John]: +human, +animate, +male, +automotive, -vehicle

61. Allen’s Model Learns associations between (i) specific verbs & argument structures and (ii) semantic representations Training set: 1200 ‘utterance types’ taken from caretaker speech in Peter corpus (CHILDES)

62. Allen’s Model Fine-grained distinction between hit, carry John kicked Mary the ball *John carried Mary the basket [kick]: +cause, +apply-force, +move, +travel, +contact, +hit-with-foot, +strike, +kick, +instantaneous-force, +hit [carry]: +cause, + apply-force, +move, +travel, +contact, +carry, +support, +continuous-force, +accompany

63. Allen’s Model Fine-grained distinction between hit, carry John kicked Mary the ball *John carried Mary the basket [kick]: +cause, +apply-force, +move, +travel, +contact, +hit-with-foot, +strike, +kick, +instantaneous-force, +hit [carry]: +cause, + apply-force, +move, +travel, +contact, +carry, +support, +continuous-force, +accompany

64. Allen’s Model Fine-grained distinction between hit, carry John kicked Mary the ball *John carried Mary the basket [kick]: +cause, +apply-force, +move, +travel, +contact, +hit-with-foot, +strike, +kick, +instantaneous-force, +hit [carry]: +cause, + apply-force, +move, +travel, +contact, +carry, +support, +continuous-force, +accompany

65. Allen’s Model Fine-grained distinction between hit, carry John kicked Mary the ball *John carried Mary the basket [kick]: +cause, +apply-force, +move, +travel, +contact, +hit-with-foot, +strike, +kick, +instantaneous-force, +hit [carry]: +cause, + apply-force, +move, +travel, +contact, +carry, +support, +continuous-force, +accompany

66. Allen’s Model Fine-grained distinction between hit, carry John kicked Mary the ball *John carried Mary the basket “This behavior shows crucially that the network is not merely sensitive to overall semantic similarity: rather, the network has organized the semantic space such that some features are more important than other.” (p. 5)

67. English John piled the books on the table. John piled the table with books. Korean Yumi-kachaek-lulchaeksang-eyssa-ass-ta. Nombook-Acctable-Locpile-Past-Dec ‘Yumi piled books on the table.’ *Yumi-kachaeksang-lulchaek-elossa-ass-ta. Nom table-Acc books-withpile-Past-Dec ‘Yumi piled the table with books.’ Verb Argument Structure Korean is more restrictive than English - conflates pile- class and pour-class.

68. 1.English 2.Korean 3.Turkish 4.Chinese 5.Japanese 6.Yoruba 7.Hebrew 8.French 9.Spanish (Castilian) 10.Spanish (Argentinian) 11.Arabic 12.Thai 13.Luganda 14.Malay 15.Hindi 16.Ewe 17.Italian 18.Brazilian Port. 19.Russian 20.Polish Verb Argument Structure Typological Survey

69. Verb Argument Structure Simple VP Structures She filled the water into the glass. She stuffed feathers into the pillow. Simple VP Structures She poured the glass with water. She piled the shelf with books. Adjectival Passives The filled water. The stuffed feathers. (*) Verb Serialization She pour-put the glass. She pile-put the shelf. EnglishKorean diff. same Korean diff. Korean diff. EnglishKorean diff. same (Kim, 1999; Kim, Landau, & Phillips, 1999) Verb class contrasts that seem to disappear in simple/frequent structures reemerge in constructions that are less frequent.

70. Verb Argument Structure Allen & Seidenberg model presents interesting case for reduced innate knowledge about linking rules If they are right that linking rules are learned from distributional analysis of common constructions –There should be more cross-language variation in verb classes –We should not find cross-linguistically consistent contrasts buried in obscure corners of the grammars of particular languages

71. ‘Constrained statistical learning’ idea –Newport & Aslin 2004 - Cognitive Psych.: only certain relations learned –Saffran 2003 - Curr. Dir. Psych. Sci.: only certain generalizations made –Saffran & Thiessen 2003 - Dev. Psych, 39, 484-494 Learning Rules –Marcus et al. 1999 - Science: generalizing beyond the training stimuli –Pena et al. 2002 - Science: importance of segmentation for generalization –Marcus et al. 2005 - Nature?: importance of speech-like stimuli –Marcus et al. 200x - Cognition: –These are simple linear patterns; language requires much more Conclusion… –Literature has clarified a number of issues regarding systematicity in language –Controlled studies of forming generalizations –Distinguish roles of statistics: representation vs. learning tool –Seidenberg & Locative verbs…? More…