Chapter 9. A Model of Cultural Evolution and Its Application to Language From “The Computational Nature of Language Learning and Evolution” Summarized by Seok Ho-Sik

© 2009 SNU CSE Biointelligence Lab 2 Contents 9.1 Background 9.2 The Cavalli-Sforza and Feldman Theory 9.3 Instantiating the CF Model for Languages One-Parameter Models An Alternative Approach Transforming NB Models into the CF Framework 9.4 CF Models for Some Simple Learning Algorithms TLA and Its Evolution Batch- and Cue-Based Learners 9.5 A Generalized NB Model for Neighborhood Effects A Specific Choice of Neighborhood Mapping 9.6 A Note on Oblique Transmission 9.7 Conclusions

9.1 Background Change in linguistic behavior of human populations must be a result of a change in the internal grammars that successive generations of humans employ.  Q: Why do the grammars of successive generations differ from each other?  Need to know: 1) how these grammars are acquired 2) how the grammars of succeeding generations are related to each other.  Then, it is possible to predict the envelope of possible changes. The applicability of the CF model to the framework for linguistic change  CF model: Cavalli-Sforza and Feldman model (cultural change model).  Introducing on possible way in which principle and parameters approach is be amenable to CF framework. © 2009 SNU CSE Biointelligence Lab 3

9.2 The Cavalli-Sforza and Feldman Theory (CF model) A theoretical model for cultural change over generation “Cultural” parameters  Being transmitted from parents to children with certain probabilities.  The mechanism of transmission is unknown – only the probabilities are known. An example of CF model  Variables with binary value L & H. The proportion of L types in the population will evolve according to the following equation The evolution (change) of cultural traits  essentially driven by the probabilities with which children acquire the traits given their parental types. © 2009 SNU CSE Biointelligence Lab 4 Table 9.1 The cultural types of parents and children related to each other by their proportions in the population. The values are for vertical transmission. b i : the probability with which a child of ith parental type will attain the trait L. p i : the probability of the ith parental type in the population. u t : the proportion of people having type L in the parental generation.

9.3 Instantiating the CF Model for Languages One-Parameter Models (1/2) In language change, the transmission probability depends upon the learning algorithm. Assumption  1. Children of parents with the same language receive examples only from the parental language.  2. Children of parent with different language receive examples from an equal mixture of both languages.  3. After k examples, children “mature”. The probability which the algorithm A hypothesizes grammar g 1 given a random i.i.d. draw of k examples according to probability distribution P. Statement 9.1 If the support of P is L 1 then and if the support of P is L 2 then © 2009 SNU CSE Biointelligence Lab 5

9.3 Instantiating the CF Model for Languages One-Parameter Models (2/2) It is possible to express the b i ’s in the CF model of cultural transmission in terms of the learning algorithm. © 2009 SNU CSE Biointelligence Lab 6 Table 9.2 The probability with which children attain each of the language types, L1 and L2 depends upon the parental linguistic types, the probability distribution P 1 and P 2 and the learning algorithm A.

9.3 Instantiating the CF Model for Languages An Alternative Approach Assumptions in previous chapters  1. The population can be divided into children and adults.  2. All children in the population are exposed to sentences from the same distribution  3. The distribution depends upon the distribution of speakers in the adult population. The evolution of s t over time ( s t : state of the population) Interpretation  If the previous state was s t, then children are exposed to sentences drawn according to  The probability with which the average child will attain a language is correspondingly provided by g. © 2009 SNU CSE Biointelligence Lab 7

9.3 Instantiating the CF Model for Languages Transforming NB Models into the CF framework NB: Niyogi and Berwick model NB update rule: CF update rule NB update rule  CF update rule The differences in the evolutionary dynamics of CF- and NB-types models  1. The evolutionary dynamics of the CF model depends upon the value of f at exactly 3 points (0, ½, 1).  2. If f is linear, then the NB and CF update rules are exactly the same. If f is nonlinear, these update rules potentially differ.  3. The CF update is a quadratic iterated map and has one stable fixed point.  4. For some learning algorithms, there may be qualitatively similar evolutionary dynamics for NB and CF models.  Essential one: NB assumes that all children receive input from the same distribution. Cavalli-Sforza and Feldman assume that children can be grouped into four classes depending on their parental types. © 2009 SNU CSE Biointelligence Lab 8

9.4 CF Models for Some Simple Learning and Algorithms TLA and Its Evolution (1/4) TLA (Triggering Learning Algorithm) working  1. Initialize: start with randomly chosen input grammar.  2. Receive next input sentence, s.  3. If s can be parsed under current hypothesis grammar, go to 2.  4. If s cannot be parsed under current hypothesis grammar, choose another grammar uniformly at random.  5. If s can be parsed by new grammar, retain new grammar, else go back to old grammar.  Go to 2. For two grammars in competition under the assumptions of the NB model,  a: -the probability that ambiguous sentences (parsable both g 1 and g 2 )are produced by L 1 speakers.  b:  k: # sentences a child receives from its linguistic environment before maturation. © 2009 SNU CSE Biointelligence Lab 9

9.4 CF Models for Some Simple Learning and Algorithms TLA and Its Evolution (2/4) Remarks  1. For k=2 and NB model, (i) for a=b, there is exponential growth to one fixed point of p* = ½. (ii) a≠b, there is logistic growth and if a ½. else if a>b, p*< ½ © 2009 SNU CSE Biointelligence Lab 10 Fig. 9.2: The fixed point for various choices of a and b for the CF model with k=2 Fig. 9.1: The fixed point for various choices of a and b for the NB model with k=2

9.4 CF Models for Some Simple Learning and Algorithms TLA and Its Evolution (3/4)  For k=2 and CF model, (i) for a=b, there is exponential growth to one fixed point of p* = ½. (ii) a≠b, there is logistic growth and if a ½. else if a>b, p*< ½ © 2009 SNU CSE Biointelligence Lab 11 Fig. 9.3: The difference in the values of p*(a,b) for the NB model and the CF model p* NB – p* CF for various choices of a and with k=2.

9.4 CF Models for Some Simple Learning and Algorithms TLA and Its Evolution (4/4)  For k  , (i) for a=b, there is no change in the linguistic composition, (ii) for a>b s t tends to 0, (iii) for a<b s t tends to 1. Thus one of the languages drives the other out and the evolutionary change proceeds to completion. In real life, a = b is unlikely to be exactly true, therefore language contact between population is likely to drive one out of existence. © 2009 SNU CSE Biointelligence Lab 12

9.4 CF Models for Some Simple Learning and Algorithms Batch- and Cue-Based Learners (1/2) For batch learners (chapter 5), the system has two stable fixed points  =  * and  = 1. There is one unstable fixed points between them  case in which NB dynamics is bistable. CF model will have only one stable fixed point. CF update rule when s t = 1, 1 is a fixed point of the CF system. If f( ½ )>1  s =1 is a stable fixed point, otherwise s=1 is unstable. In contrast, the NB system is always stable at 1. © 2009 SNU CSE Biointelligence Lab 13

9.4 CF Models for Some Simple Learning and Algorithms Batch- and Cue-Based Learners (2/2) For cue-based learner NB dynamics  In a regime, NB dynamics has only one stable fixed point  =0.  There is another regime where NB dynamics has two stable fixed points  =0 and  =  * >0 CF dynamics  CF dynamics has an fixed point at s = 0. Stable fixed point f( ½ ) ½  s = © 2009 SNU CSE Biointelligence Lab 14

9.5 A Generalized NB Models for Neighborhood Effects CF model: sentences from different distribution. NB model: sentences from the same distribution. Generalization  Key idea: heterogeneous communities speakers often tend to cluster in linguistically homogeneous neighborhoods.  children’s input sentences are depending on their location in the neighborhood.  h -mapping: mapping from neighborhood to  -type  proportion of L1 speakers that an  -type child is exposed to.  The percentage of speakers of L 1 f(  ): a probability of attaining the grammar L 1 © 2009 SNU CSE Biointelligence Lab 15 P h (  ): distribution of 

9.5 A Generalized NB Models for Neighborhood Effects A Specific Choice of Neighborhood Mapping For figure 9.6 Update rule 1. The linear map implies an exponential growth to a stable fixed point 2. s* =1 requires  very unlikely  no language is likely to be driven out of existence completely. In contrast, NB and CF models result in extinction if a ≠ b. Remarks  1. h is not fixed function. It changes from generation to generation.  2. The population of mature adults is always organized into two linguistically homogeneous neighborhood in every generation. © 2009 SNU CSE Biointelligence Lab 16

9.6 A Note on Oblique Transmission Oblique transmission: the effect that members of the parental generation at large have on the transmission of cultural traits. Stage 1: children acquire on the basis of preliminary exposure to their parents. Stage 2: juvenile acquire a “mature” state. One computes the probability with which trait transitions occur. If  1,  2 are known  compute the dynamics of H types in the population. © 2009 SNU CSE Biointelligence Lab 17