1. EUROPEAN SOCIETY FOR THE STUDY OF COGNITIVE SYSTEMS 20-22 August 2007, Groningen, The Netherlands ABSTRACT ASCENT OF INTELLIGENCE Gesture and Language : Mind and Body Robin Allott

2. The acquisition of language was the turning-point for the evolutionary separation of humans from apes. From this flowed the ascent of human intelligence with the ratcheting up of human mental and cultural advance as a result of interaction between individual variations in brain structure and development and continually advancing complexity of the social, technological and cultural environment (a manifestation of the Baldwin effect).

3. Both body and brain were involved in the acquisition and advance of language, phylogenetically and ontogenetically. The relation between mind-state and body-state was manifested not only in the patterning of emotion (the James/Lange theory) but also in the selection of words (articulatory programs) to match patterns of perception, both of external objects and also innate attitudinal patterns, "mental" structures, which went to form both language syntax and motor syntax (cf. Karl Lashley, Kant).

4. The key aspect of the motor theory of language is that words, speech and language are the outcome of an exaptation of the motor control system, that is, a direct relation between aspects of the motor cortical system and the characteristic features of lexicon and syntax.

5. Progress in neuroscience, and particularly recent experimental research using fmri and other techniques, has provided material support for the motor basis of language.

6. RECENT RESEARCH (1) A larynx area in the human motor cortex Brown Ngan Liotti Cerebral Cortex July 25 2007 A human evolutionary novelty perhaps related to emergence of voluntary control of vocalisation (2) The organisation of behavioral repertoires in the motor cortex 2006. Graziano M. Annu. Rev of Neurosci. 29. (2) "When Language Meets Action: The Neural Integration of Gesture and Speech. 2006 Willems Ozyurek Hagoort Cereb. Cortex 2006 Dec 11 (Epub) fMRI evidence that speech and gesture share a high-level neural integration system. (3) "Speech and Gesture share the same communication system" 2006 Bernardis Gentilucci Neuropsychologia 44 178-190. Experiment suggests that word and gesture are related at the levels of execution and processing with implications for the evolution of language. (4) Comparison of the neural systems underlying speech and non-speech vocal utterances 2000 Jurgens. Existence in humans (and not in other primates) of a direct connection between the motor cortex and the laryngeal motoneurons.

7. OTHER RELEVANT MATERIAL (6) Functional links between motor and language systems 2005 Pulvermuller Hauk Nikulin Ilmoniemi Eur J Neurosci 3 1793-7. TMS experiment showing specific links between action and language systems during lexical processing. (8) Complex movements evoked by microstimulation of precentral cortex. 2002 Graziano Taylor Moore Neuron 34, 841-851. (9) The cortical control of movement revisited. Graziano Taylor Moore Cooke Neuron 36, 349-362. "One possibility is that the mechanisms for speech were built on a preexisting mechanism for motor control. (7) Origin of speech: The motor route Holden 2004 Science 303:1316- 1319. Abundant behavioral evidence for an intimate connection between language and motor abilities.

8. MORE SPECULATIVE (5) Hypoglossal canal and the origin of human vocal behavior 1998 Kay Cartmill Balow PNAS 95 5417-5419. The much larger canal in humans than in apes or australopithecus makes possible richer motor innervation of the tongue and so made possible language as a uniquely human ability. (10) FoxP2 gene 2001 Varga-Khadem et al. Nature 413 519-523. Motor control and language implications. (11) Language within our grasp. 1998. Rizzolatti, G. and M. Arbib. Trends in Neuroscience 21 188-194. Mirror neurons make possible empathy and imitation and so provide a basis for the evolution of language. (12) EvoDevo (13) Baldwin effect

9. THE ORIGIN OF WORDS FROM THE IMITATION OF SOUNDS SHAPES AND ACTIONS THEN TRANSDUCED INTO ARTICULATORY PROGRAMS BY MOTOR EQUIVALENCE

10. Motor equivalence has been the central process in the origin and functioning of language and the acquisition of words.

11. EVIDENCE ? EXAMPLES ? EXPERIMENT ?

12. THOUGHT EXPERIMENTS ? Or MATERIAL EXPERIMENTS ? OR MIND EXPERIMENTS ?

13. The sound [the word] is not a directly imitative sign but indicates a quality which the sign and the object have in common.... sounds which partly independently and partly in comparison with others produce an impression which to the ear is similar to that which the object makes upon the mind. Humboldt

14. Parallelisms of word and gesture (dual expressions of meaning) can be made overt by specific controlled mind/brain operations

15. LANGUAGE AND THE ASCENT OF INTELLIGENCE

16. INTELLIGENCE ? Brain size and structure ? Baldwin effect ? EvoDevo ?

17. BRAIN SIZE The inescapable fact is that brains, and particularly human brains, have much increased in size in the course of evolution. The increase in size must have brought survival benefits and for humans it surely means that however intelligence is measured greater size has moved in step with greater intelligence - though at the individual level the correlation is not exact.

18. BUT … all the day-to-day routines of bodily existence require very little neural mass. Ants, bees, mice, birds, dinosaurs, manage, or managed, very well, with small, or extremely small, brains (the ratio of brain to body may be more significant, for example, for mice or other small rodents). ]

19. So why and how did humans come to acquire a brain-mass much greater than is needed for routine bodily functions ? From home habilis to modern homo sapiens the brain grew from an average 750 cc. to 1350-400 cc. [No known comparable rate of increase in brain size in any other species at any time in the history of life on earth. Plotkin 1996]

20. HYPOTHESES 1. Social complexity 2. Foraging strategies 3. Language development

21. There has been intense and long-enduring discussion of these issues and many unresolved arguments with much speculation and little useful evidence. Here it is only possible to make a few comments:

22. SOCIAL COMPLEXITY Whether, for humans, social complexity favouring larger brains could have developed without some form of language is unclear. On the other hand, there are animals, ants, bees, termites, which manage complex societies with minuscule brains. The case is made mainly in terms of ape behaviour - the orangutan, one of the more intelligent animals is solitary.

23. FORAGING This account has been developed mainly by specialists in primate behaviour. It may go some way to account for the relatively large brains of chimpanzees and gorillas but has little explanatory value for the remarkable near-double increase in the human brain, even when associated with the somewhat implausible development of the idea of Machiavellian intelligence - that deceiving group members was one of the necessary aspects of increased intelligence.

24. LANGUAGE That the remarkable increase in human brain size (unmatched by any similar rate of increase in other animals) should have some relation to the equally remarkable (unmatched by other animals) evolutionary human acquisition : language, seems an obvious and plausible hypothesis.

25. But the debate, confusion and uncertainty about the process by which humans invented, acquired or developed language, or languages, still rages, after 2500 years. The question about the evolutionary relation of human brain-size and language goes with the unsettled question about the brain or social processes making possible another remarkable achievement, the untutored, extensive and rapid acquisition of language, complex syntaxes and massive lexicons, by children

26. EVO-DEVO Evolutionary developmental biology BALDWIN EFFECT Behaviour evolution interaction

27. A NEW EVO-DEVO BALDWIN EFFECT The accepted evolutionary account of the Baldwin effect was that humans, and other animals, by changing their behaviour changed their environment and so created novel potentialities for natural selection which could operate in succeeding generations, making it possible for culture to modify evolution. With the recent growth of the new discipline Evo-Devo concerned with the relation between developmental and evolutionary processes, a new application or understanding of the Baldwin effect in relation to development of the brain becomes possible.

28. CELL DEATH In the development of the brain, many more neurons are produced than are ultimately needed to create the mature brain. The neurons are thinned out by programmed cell death; the initial supply of neurons (twice as many as eventually survive) are in competition to establish appropriate connections. Those which do not get the necessary access to a source of NGF (nerve growth factor) die, this Neural Darwinism (in a sense completely different from Gerald Edelmans term) is natural selection at work.

29. THE DYING CELL 3 cells compete for Nerve Growth Factor. 2 win and the 3rd dies (repeating loop)

30. Through cell death each individual brain is sculpted to match the environment in which the brain develops (both fetally and for an extended period after birth). Cell death continues shaping the brain long after birth. For all types of cells, from 8-14 the average child loses approx. 20-39 billion cells a day. The average adult loses approx.50-70 billion a day.

31. CELL DEATH AND EVOLUTION

32. Cell death can now be recognised as an important link between brain and environment, operating through life to permit experience to shape the brain. It is via cell death that the relation between brain-size and culture including language can be understood. A key point in the operation of this process in he developing brain is individual brain variation. Each brain is unique in many ways, including its size and the number of neurons which have to find appropriate connections to survive.

33. BRAIN INDIVIDUAL DIFFERENCE Natural selection can come into play on differences in brain structure derived from the culture. Increased brain size means the availability of more neurons (and more neuronal inter-connections) available to match the extending experience of the individual- and so increase the selective advantage of the individual in a changing culture. A virtuous circle is established with the better adapted and no doubt larger- brained individuals playing a larger part in manipulating the environment in which the next generation will have to face natural selection and in their turn have their brains shaped by experience.

34. Language in the group will account for an ever-larger segment of total cultural input to the brain and will also act as a powerful instrument in shaping the social system. A ratchet effect is established which goes to promote a persisting increase in brain-size (and skull-size co- ordinated by allometry with brain-size) until the skull-size hits the anatomical constraint of birth-canal size..

35. SKULL AND BRAIN SIZE The brain no doubt keeps on growing in complexity to accommodate continuing language and other cultural changes but the growth must take the form of internal re-arrangement to produce the elaborate infolding seen in the modern human brain.

36. LEXICON INPUT But how does language drive this? The major new language input to the brain is through a rapidly expanding lexicon. The growing lexicon, on the motor theory of language, requires that each word creates a articulatory motor program (linked to and derived from the visual or action percept) The influx of novel words involves an increasing demand for neurons and neuronal connections. Baldwin Evo-devo is the form in which language drives growth in the size of the human brain in response to the acquisition of words.

37. SYNTAX ? Additional demand for neurons and connections to accommodate syntax is less significant. Language syntax can rely substantially on pre-existing organisation of the motor and visual systems, motor syntax and vision syntax, and the neurons and connections serving these.

38. SPEECH But of course this only goes so far in explaining the evolutionary role of language. Before the Baldwin Evo-Devo process can begin to operate to increase brain-size, there must already be language and language-related culture. There must already be words. The central question remains: WHY and HOW could speech and language have got going for humans at all? Why humans and not dogs or apes? The most plausible possibility is, as Jan Wind suggested long ago, not that there was some massive mutation but a continuing process of cerebral reorganisation. Relatively minor changes, well within the scope of inherent brain plasticity, could have made speech possible.

39. WHAT FORM COULD THESE CHANGES HAVE TAKEN? Research papers listed earlier offer some possibilities:

40. MOTOR CORTEX FOR SPEECH Jurgens: Neuroanatomically, the step from genetically determined controlled vocal patterns is associated with the emergence of a direct connection between the motor cortex and the laryngeal motoneurons, a connection lacking in subhuman primates.

41. INCREASED NERVE SUPPLY Kay Cartmill Balow: Hypoglossal canal and the origin of human vocal behavior The hypoglossal canal (which carries nerves controlling tongue movements) is much larger in humans than in other primates or in australopithecus. The larger canal is adapted to carry a much richer motor innervation of the tongue and so to make possible language as a uniquely human ability.

42. CONTROL OF THE LARYNX Brown Ngan Liotti A larynx area in the human motor cortex A human evolutionary novelty perhaps related to emergence of voluntary control of vocalisation.

43. MOTOR CONTROL Pulvermuller Hauk Nikulin Ilmoniemi Functional links between motor and language systems. TMS experiment showing specific links between action and language systems during lexical processing. Graziano Taylor Moore Cooke The cortical control of movement revisited. One possibility is that the mechanisms for speech were built on a preexisting mechanism for motor control.

44. BUT WHERE DID THE WORDS COME FROM ? Some, or all, of the listed cortical changes could have made speech possible for humans (but not for other primates). But for the Baldwin Evo-Devo effect to operate there had to be words. Where did they come from? Where does any individual word come from? Herder said that it was totally impossible that words should be arbitrary, that someone should invent say the word GREEN out of the top of his head for the distinctive colour Green. If he picked a set of speech sounds at random, say POGGLE, to mean Green, why should others accept and understand him Even more impossible, how could anyone arbitrarily invent the words IF, MIND, SAD, THINK - there is nothing to point to for these words; there is no sound or shape to imitate.

45. LANGUAGE WITHIN OUR GRASP Rizzolatti and Arbib argued that the discovery of mirror neurons linking responsive motor programming in the brain of an observer with observed motor patterning of action of another individual, could have been the basis for the evolution of language. The mirror neurons could have made, and still make, imitation possible, including imitation of gesture. Arbib argues that the ability to imitate is a key innovation, a neurobiological missing link for the hypothesis that primitive forms of communication based on manual gesture preceded speech in the evolution of language. A possible evolutionary path from manual skills to language. Gallese says the discovery of mirror neurons may provide a new, though still sketchy, neurobiological basis to account for the emergence of language.

46. SO WHAT CAN IMITATION DO IN THE EMERGENCE OF LANGUAGE ? Some things can be indicated by gestures, a tree, the sky, a direction, up or down, come and go, high and low. Gesture can be used to point to things, to indicate hearing, eating or drinking, etc. But for many things manual gesture is inadequate: colours - white black red, sounds, different animals - what would be a manual gesture for a horse, a fish, a rose, a cabbage? Imitation must extend much wider than manual gesture, for example to animal cries, the noise of the wind or rain, thunder, lightning, the sea.

47. IMITATION IN THE BRAIN Imitation v mimicry - What matters is stored imitation (requiring commitment of neurons and interconnecting fibres) in the form of a link between word and imitated act, sound or shape so that there can be ready access to the word and what it refers to. Words are anchored in the motor patterning and are expressible as bodily and articulatory gesture. Deacon quoted by Gerhard: the everyday miracle of word meaning and reference. So each individual word was an arbitrary invention with an equally arbitrary linking to what it related to! With Herder and Humboldt, surely not.

48. HOW DOES THE WORD GET LINKED TO WHAT IT REFERS TO ? How was each gesture invented? Was each gesture as arbitrary as traditional linguistics says that each word is? Clearly not. A gesture is patterned by the action seen, the shape of what is seen, the sound hear, for a vocal gesture. Mirror neurons may allow transfer of the pattern of an action and the ability to reproduce the action as a gesture (not in any way arbitrary) - but imitation has to be possible for much beyond perceived action.

49. HOW WORDS WERE FORMED The process by which words were formed was the inverse of the process by which gestures and sounds can be generated from existing word-forms - a reverse application of motor equivalence. On seeing some one hitting something, the action patterning was by motor equivalence converted into articulatory patterning to produce a speech-sound structure, a word, directly related to the action patterning seen. Similarly on hearing an animal sound, the typical sound of a cat or a lion, the sound-patterning is transduced by motor equivalence to form a word whose structure is derived from the sound heard.

50. IN BRIEF Gestures of all kinds were generated by imitation of actions, shapes and sound. These were stored as motor programs before humans acquired speech When cerebral reorganisation provided new direct connections between the motor cortex, the tongue and the larynx, there was a great increase in the innervation of the articulatory apparatus generally. The motor programs from gestural origins were transduced automatically into words structured by the gestural programs. The meanings of words were automatically linked to the action, sound and shape percepts to which the gestures referred.