Emergent Adaptive Lexicons Luc Steels 1996 Artificial Intelligence Laboratory Vrije Universiteit Brussel Presented by Achim Ruopp University of Washington

Agenda Introduction to Emergence and Self- Organization Steel’s Experiments on the emergence of the lexicon Discussion

Excursion: Emergence and Self-Organization A system exhibits emergence when –There are coherent emergents at the macro-level –That dynamically arise from interactions between parts a the micro-level –Emergents are novel w.r.t. individual parts of the system

Excursion: Emergence and Self-Organization Definition –A dynamical and adaptive process –Where systems acquire and maintain structure themselves –Without external control Combining emergence and self-organization

Origins of Language Still unknown Chomsky’s hypothesis –Species-specific innate language ability –Refinement by parameter setting process –Some support by experimental simulation via neural networks Alternative hypothesis –Language as an emergent phenomenon As a mass-phenomenon Spontaneously forming/becoming more complex

Steel’s Experiments Motivated by symbol grounding problem Experiments on robotic and software agents –Grounded meaning creation –Lexicon formation ← Focus of paper –Syntax –Emergent phonology

Experimental Model Definitions Features Set of agents A ∀ a ∈ A ∃ set of features F a = {f 0,…,f n } A feature f i consists of attribute-value pairs –Examples: (weight light) (size tall) Distinctive feature set –Subset of F a distiguishing agent a from all agents in a group B Filtered subset C K,M = {a|K ⊂ F a } –K is a set of features –M is a set of agents

Experimental Model Definitions Lexicon Word –sequence of letters drawn from a shared alphabet Utterance –Set of words –Word order does not play a role Lexicon L –A relation between feature sets and words –A single word can have several associated feature sets –A feature set can have several associated words

Experimental Model Definitions Lexicon Each agent a has lexicon L a –Initially empty Feature set of a word: F w,L Cover functions –cover(F,L): set of utterances U: ∀u ∈ U: {f|f ∈ F w,L and w ∈ u} –uncover(u,L): set of features F: F = {f|f ∈ F w,L and w ∈ u}

Coherence through Self-Organization Agents can –Create new words and associate them with a feature set –Form new associations between a word and a feature set Key to self-organized coherence of the lexicon –Agents participate in communication –Record the success of particular word-meaning pairs –Agents (re-)use words that led to high communication success in the past

Language Game Dialog between two agents – Speaker and Hearer Dialog topic –Other Agent –Chosen by extra-linguistic means (“pointing”) Speaker and hearer identify possible distinctive feature sets of topic Speaker –Selects distinctive feature set –Translates to words using cover function Hearer –Interprets utterance using uncover function –Compares interpretation to expectation –Uses game to Learn part of the language Check if right meaning is associated with the right words

Language Game Possible Outcomes 1.No differentiation possible 2.Speaker does not have a word –May create new word 3.Hearer does not have a word –Can associate word –Cannot disambiguate when multiple distinctive feature sets

Language Game Possible Outcomes 4.Speaker and hearer know word Meanings are compatible with situation Sense-ambiguity possible Meanings not compatible with situation No communicative success

Experimental Results One-word Utterances Typical experimental setup (5 agents, 10 meanings, 4000 language games) Leads to communicative success soon Average communicative success Number of language games (scale 1/20)

Experimental Results One-word Utterances Single meanings soon converge on one word form (10 agents, 5 possible words, 1 meaning) Average communicative success Time

Experimental Results Multiple Word Utterances In case the distinctive feature set of the topic contains multiple features Can be used by hearer to “guess” meaning of unknown words

Conclusions Self-organization is effective mechanism for achieving coherence Side-effects of lexicon formation –Synonymy –Ambiguity –Multiple-word sentences

Discussion Supports the notion that absolute synonymy does not exist –“After about 4000 language games the lexicon stabilizes as all distinctions that need to be made have been lexicalized” Are the presented “linguistic” results an artifact of the experimental setup? I.e. in how far does this experiment reflect the real world? –E.g. multi-word sentences Can the results just be explained by basic communication theory? Language as an emergent phenomenon –Zipf’s law regarding multiple meanings –Self-organized criticality/highly optimized tolerance

References Brighton, H., Selina, H.; Introducing Artificial Intelligence; 2003; Icon Books Ltd.; ISBN De Wolf, Tom; Holvoet, Tom; Emergence Versus Self-Organisation: Different Concepts but Promising When Combined; 2005; In Engineering Self Organising Systems: Methodologies and Applications, Lecture Notes in Computer Science, volume 3464, pp 1-15 Steels, L.; Emergent Adaptive Lexicons; 1996; In: Maes, P. and Mataric, M.J. and Meyer, J.-A. and Pollack, J. and Wilson, S.W. (eds) From Animals To Animats 4: Proceedings of the Forth International Conference on Simulation of Adaptive Behavior, SAB'96, Complex Adaptive Systems, pp , Cambridge, MA: The MIT Press Zipf, G. K.; The Meaning-Frequency Relationship of Words; Journal of General Psychology 33, 251–256 (1945).