1. Hao Wang, Toben Mintz Department of Psychology University of Southern California

2. The Problem of Learning Syntactical Categories Grammar includes manipulations of lexical items based on their syntactical categories. Learning syntactical categories are fundamental to the acquisition of language.

3. The Problem of Learning Syntactical Categories Nativist approach Children are innately endowed with the possible syntactical categories. How to map a lexical item to its syntactical category or categories? Empirical approach Children have to figure out the syntactical categories in their target language, and assign categories to lexical items. There is no or little help from syntactical constraints.

4. Approaches Based on Semantic Categories Grammatical Categories correspond to Semantic/Conceptual Categories (Macnamara, 1972; Bowerman, 1973; Bates & MacWhinney, 1979; Pinker, 1984) object  noun action  verb But what about action, noise, love to think, to know (Maratsos & Chalkley, 1980)

5. Grammatical Categories from Distributional Analyses Structural Linguistics Grammatical categories defined by similarities of word patterning ( Bloomfield, 1933; Harris, 1951) Maratsos & Chalkley (1980): Distributional learning theory lexical co-occurrence patterns (and morphology and semantics) the cat is on the mat cat, mat

6. Grammatical Categories from Distributional Analyses Patterns across whole utterances (Cartwright & Brent, 1997) My cat meowed. Your dog slept. Det N X/Y. Bigram co-occurrence patterns (Mintz, Newport, & Bever, 1995, 2002; Redington, Chater & Finch, 1998) the cat is on the mat

7. Probabilistic Bigram Co-Occurrence Patterns X was - 20% there - 5% is - 30% wants - 15% seeing - 7% … the - 57% a - 25% blue - 0% silly - 5% your - 7% … Y was - 20% has - 30% is - 30% belongs - 2% around - 5% … the - 45% a - 35% blue - 5% silly - 0% your - 10% …

8. Frequent Frames (Mintz, 2003) Frames are defined as “two jointly occurring words with one word intervening”. “would you put the cans back ?” “you get the nuts.” “you take the chair back. “you read the story to Mommy.” Frame: you_X_the

9. Sensitivity to Frame-like Units Frames lead to categorization in adults (Mintz, 2002) Fifteen-month-olds are sensitive to frame-like sequences (Gómez & Maye, 2005)

10. Other Motivation for Frames Verb learning in children can be facilitated by frequent frames (Childers & Tomasello, 2001) Aspects of verb meaning carried by verb frame, linguistically defined (Gleitman, 1991; Gillette, Gleitman, Gleitman, Lederer, 1999; etc.)

11. Distributional Analyses Using Frequent Frames (Mintz, 2003) Six corpora from CHILDES (MacWhinney, 2000). Analyzed utterances to children under 2;6. Accuracy results averaged over all corpora.

12. Limitation of the Frequent Frame Analyses Requires two passes through the corpus Step 1, identify the frequent frames by tallying the frame frequency. Step 2, categorizing words using those frames. Tracks the frequency of all frames E.g., approximately 15000 frame types in one of the corpora in Mintz (2003).

13. Goal of current study Provides a psychological plausible model of word categorization Children possesses limited memory and cognitive capacity. Human memory is imperfect. Children may not be able to track all the frames he/she has encountered.

14. Features of current model It processes input and updates the categorization frames dynamically. Frame is associated with and ranked by a activation value. It has a limited memory buffer for frames. Only stores the most activated 150 frames. It implements a forgetting function on the memory. After processed a new frame, the activation of all frames in the memory decreased by 0.0075.

15. Child Input Corpora Six corpora from CHILDES (MacWhinney, 2000). Analyzed utterances to children under 2;6. Peter (Bloom, Hood, Lightbown, 1974; Bloom, Lightbown, Hood, 1975) Eve (Brown, 1973) Nina (Suppes, 1974) Naomi (Sachs, 1983) Anne (Theakston, Lieven, Pine, Rowland, 2001) Aran (Theakston et al., 2001) Mean Utterance/Child: ~17,200 MIN: 6,950 ; MAX: 20,857

16. Procedure The child-directed utterances from each corpus was processed individually Utterances were presented to the model in the order of appearance in the corpus Each utterance was segmented into frames “you read the story to Mommy” you read the read the story the story to story to Mommy

17. Procedure continued… you read the read the story the story to story to Mommy Memory ActivationFrame 1.0000you_X_the 1.0000read_X_story 1.0000the_X_to 1.0000story_X_Mommy

18. Procedure continued… The memory buffer only stores most activated 150 frames. It becomes full very quickly after processing several utterances. Memory ActivationFrame 1.0000you_X_the 1.0000read_X_story 1.0000the_X_to 1.0000story_X_Mommy 1.0000to_X_it 1.0000the_X_on ……

19. Procedure continued… “you put the” Frame: you_X_the Look up you_X_the frame in the memory Increase the activation of you_X_the frame by 1 Re-rank the memory by activation Memory ActivationFrame 1.0000you_X_the 1.0000read_X_story 1.0000the_X_to 1.0000story_X_Mommy 1.0000to_X_it 1.0000the_X_on ……

20. Procedure continued… “you have a” Frame: you_X_a Look up you_X_a frame in the memory story_X_Mommy < 1 Remove story_X_Mommy Add you_X_a to memory, set the activation to 1 Re-rank the memory by activation Memory ActivationFrame 1.0000you_X_the 1.0000read_X_story 1.0000the_X_to 1.0000to_X_it 1.0000the_X_on 0.8175story_X_Mommy ……

21. Procedure continued… A new frame not in memory The activation of all frames in memory are greater than 1 There is no change to the memory. Memory ActivationFrame 1.0000you_X_the 1.0000read_X_story 1.0000the_X_to 1.0000to_X_it 1.0000the_X_on 0.8175story_X_Mommy ……

22. Evaluating Model Performance Hit: two words from the same linguistic category grouped together False Alarm: two words from different linguistic categories grouped together Upper bound of 1

23. V ADV V Accuracy Example Hits: 10 False Alarms: 5 Accuracy:

24. Ten Categories for Accuracy Noun, pronoun Verb, Aux., Copula Adjective Preposition Adverb Determiner Wh-word Negation -- “not” Conjunction Interjection

25. Averaged accuracy across 6 corpora Accuracy Eve0.782019 Peter0.803401 Anne0.872820 Aran0.860191 Nina0.828753 Naomi0.773230 Average0.820069

26. The Development of Accuracy Accuracy are very high and stable in the entire process

27. Compare to Frequent Frames After processing about half of the corpus, 70% of frequent frames are in the most activated 45 frames in memory.

28. #w2 typew2 tokenActivationFrame 09351326.25225what_X_you 120230205.151you_X_to 270203178.16525you_X_it 32711590.7135you_X_a 44411590.379you_X_the 5311085.2665are_X_doing 6511085.10525what_X_that 71510883.2965you_X_me 8389065.2905to_X_it 928965.132would_X_like 10118661.1075why_X_you Memory of Final Step of Eve Corpus

29. Stability of Frames in Memory Big changes of frames in memory in early stage, but become stable after processing 10% of the corpus

30. Summary After processed the entire corpus, the learning algorithm has identified almost all of the frequent frames by highest activation. Consequently, high accuracy of word categorization is achieved. After processing fewer than half of the utterances, the 45 most activated frames included approximately 70% of frequent frames.

31. Summary Frames are a robust cue for categorizing words. With limited and imperfect memory, the learning algorithm can identify most frequent frames after processing a relatively small number of utterances. Thus yield a high accuracy of word categorization.