Language Evolution and Change Presented by Brianna Conrey Complex Adaptive Systems Seminar February 27, 2003

Timescales of language evolution –(Kirby & Hurford, 2002; Parisi & Cangelosi, 2002) –Ontogeny Learning/Language development in individual –Glossogeny Cultural evolution/Historical change –Phylogeny Biological/species-level evolution

Constraints on language evolution “Language as an organism” –“Whereas humans can survive without language, the opposite is not the case. Thus, language is more likely to have adapted itself to its human hosts than the other way around” (Christiansen et al., 2002).

Constraints on language evolution Speaker capabilities –Cognitive: perception, memory, learning –Motor/physiological: production/articulation Speaker interactions –Space, both physical and social

Models of language evolution and change –Usually focus on one “level” of language (phonology, syntax, or lexicon) or try to bridge gaps between levels –Either an ontogenetic or glossogenetic timescale –Typically minimize number of agents interacting and/or role of space –Adults often don’t exhibit language change, which is assumed to take place mostly at the time of language acquisition

Some specific models… Naming games: Language learning –Steels (1997; 2002) Evolution of compositional language “in a community” –Kirby & Hurford (2002); Parisi & Cangelosi (2002) Emergence of dialects –Nettle (1997); Livingstone (2002) Self-organization of vowel systems –de Boer (2000; 2002)

Iterated Learning Model (ILM) Kirby & Hurford, 2002; Kirby, 2001 I-language and E-language (Chomsky, 1986) Components of model: –Meaning and signal spaces (here both 8-bit binary) –Language-learning and language-using (adult) agents Unidirectional networks map signals to meanings

ILM, continued Each iteration has one learner and one adult –At end of cycle, learner becomes adult for new cycle Initially no I-language for adult “Obverter learning strategy” for signal production –Find signal that maximizes hearer’s chance of understanding intended meaning; assume hearer’s mapping approximates own Training through backpropagation

“8-bit” results Type of behavior depends on training-set size: –Small: inexpressive, unstable –Large: completely expressive and stable –Medium: also completely expressive and stable, but reaches this state more quickly Languages from large training sets have essentially random mappings, but medium training sets have highly structured mappings –Why? –Is this significant for real language? Nowak, Komarova, & Niyogi (2001) have a similar result –Number of input sentences necessary to learn “correct” grammar is also medium »Accuracy is too low with small number, and learning period is too long with large number

Emergence of recursive compositionality Simulations use predicate logic for representing meanings; strings of characters for signals –Ex.: loves(mary,john) marylovesjohn Heuristic-drive inductive learning algorithm –First incorporate rule, then search for generalizations over pairs of rules “Random invention” for new strings –Both speaker and hearer add invented strings to linguistic knowledge

Emergence of recursive compositionality (syntax)

Modeling irregularity Languages are not completely compositional, but have some irregularities “Principle of least effort” –Shortest string produced for a given meaning –Small probability of dropping characters from utterance Frequency –Use non-uniform probability distribution over meaning space inspired by Zipf’s law (word usage is inversely proportional to its frequency rank)

Frequency correlates with irregularity

This is also what happens in natural languages –English verb frequency example In simulation, irregular forms only persist when they have high frequency

Kirby & Hurford’s conclusions “Bottleneck” at point of language transmission means that generalizations have transmission advantage historically Importance of “obverter property” Cool result: when agents generalized part of time and rote-memorized the rest of the time, general rules still fixed in language –A regular and consistent E-language does not imply that I-language is as clean…

ILM Issues Ecological validity –“Community”? –Cognitive and motor constraints not really considered (I-language and E-language are implemented fairly abstractly) –What linguistic levels (phonology, morphology, lexicon, syntax) are being modeled? Compositionality as an evolutionary advantage to “language itself”?

Self-organization of vowel systems de Boer (2000, 2002) Concerned with actual linguistic data on vowel systems Attempts to account for “universal” characteristics of vowel systems through functional explanations –E.g., articulatory ease, acoustic distinctiveness, process of learning “Optimization” as an emergent property of the system rather than a property of individual speakers

Vowel systems model Each agent has three parts (S, D, V) –S: synthesis function Mapping from possible articulations to possible acoustic signals; includes some noise Output is formant frequencies of vowels, which are what is exchanged during communication –V: vowel prototype set Initially empty; not fixed in size Based on idea of categorical perception of speech sounds –D: function calculating distance between heard sound A and each vowel prototype Recognized vowel is one that minimizes D

Development of system Imitation game –Two agents picked at random from population (N=20) to be initiator and imitator –Initiator produces sound to be imitated –Imitator finds closest prototype to this sound –Initiator communicates (“non-verbally”) whether this was the intended sound –Imitator can then alter vowel inventory

Changes an agent can make

Model results Conforms well to data from human languages Dotted line = emerged systems; solid line = real systems

Emergent five-vowel systems

Vowel system model issues Good for isolated vowels, but what about more complex signals? other aspects of language? Agent interactions still limited Convergence on one vowel system within a community, but in reality dialects of a single language often vary most in their vowel systems, even in number of vowels

The emergence of dialects Livingstone, 2002 Phenomenon of dialect continua Goal of model to show that linguistic diversity could emerge even without social motivation, which other models (e.g. Nettle, 1997) have assumed to be necessary –Spatial organization only

The emergence of dialects Agents in a single line Implementation of de Boer’s model of vowel systems, with additional constraint of communication only within neighborhood (neighborhood size is a model parameter) Results in formation of dialect continua

Dialect emergence model issues Spatial organization is not very complex Spatial factors are important, but social factors do also seem to play a role in linguistic diversity –E.g., AAVE; Labov’s study of Martha’s Vineyard

General points for discussion What kinds of issues do these models address well? fail to address? –Interaction of social and spatial factors in language change –Continuous language change: adult language changes, too! –Idiolects and their relationship to overall notions of “dialect” and “language”

General points for discussion Bridge between phonology and syntax? –Linguistic “levels” (phonology, morphology, syntax) –Nowak & Komarova (2001) Phonology and syntax are two combinatorial levels –de Boer (2002) Ability to learn temporal sequences may be connection between speech (i.e. phonology) and syntax –Kirby & Hurford (2002; Kirby, 2001) Hard to pinpoint what “level” their frequency effects describe –Bybee (2002) Frequency effects: articulatory reductions found across morpheme boundaries; word-chunking