Words in the Brain: Six Hypotheses Ling 411 – 13

Hypothesis I: Functional Webs A word is represented as a functional web Spread over a wide area of cortex  Includes perceptual information Relating to the meaning  As well as specifically conceptual information For nominal concepts, mainly in »Angular gyrus »(?) For some, middle temporal gyrus »(?) For some, supramarginal gyrus  As well as phonological information Temporal, parietal, frontal WORDS IN THE BRAIN

Part of the functional web for fork V M C T P PA PP A Set of Hypotheses All of these lines represent bidirectional connections (hence, re- verberation)

Compare Pulvermüller’s version Friedemann Pulvermüller, The Neuroscience of Language, 2002 Phonological representation: a distributed representation in the perisylvian area Meaning of a verb Meaning of a visual object

Hypothesis II: Nodes as Cortical Columns  Information is represented in the cortex in the form of functional webs (Hypothesis I) A functional web is a network within the cortical network as a whole  consisting of nodes and their interconnections connections represented in graphs as lines  Nodes are implemented as cortical columns  The interconnections are implemented as inter- columnar neural connections and synapses Axonal fibers Dendritic fibers WORDS IN THE BRAIN

The node as a cortical column  The properties of the cortical column are approximately those described by Vernon Mountcastle Mountcastle, Perceptual Neuroscience, 1998  Additional properties of columns and functional webs can be derived from Mountcastle’s treatment together with neurolinguistic findings Method: “connecting the dots”  Hypothesis IV: (Coming Soon!)

Quote from Mountcastle “[T]he effective unit of operation…is not the single neuron and its axon, but bundles or groups of cells and their axons with similar functional properties and anatomical connections.” Vernon Mountcastle, Perceptual Neuroscience (1998), p. 192

Findings relating to columns ( Mountcastle, Perceptual Neuroscience, 1998)  The column is the fundamental module of perceptual systems probably also of motor systems  This columnar structure is found in all mammals that have been investigated  The theory is confirmed by detailed studies of visual, auditory, and somatosensory perception in living cat and monkey brains

A Memory Experiment (Pulverműller 2002: 26-27)  Performed with macaque monkeys  Delayed matching – monkey must remember Monkey must keep in mind the shape or color of an object and perform a matching response after delay of several seconds  Neural activity detected in frontal and temporal lobes  Temporary lesion of frontal or temporal area leads to impaired stimulus specificity in other area  Supports the hypothesis of a functional web including sites in frontal and temporal areas Why?

Pulvermüller’s line of reasoning 1. “If neurons in the functional web are strongly linked, they should show similar response properties in neurophysiological experiments. 2. “If the neurons of the functional web are necessary for the optimal processing of the represented entity, lesion of a significant portion of the network neurons must impair the processing of this entity. This should be largely independent of where in the network the lesion occurs. 3. “Therefore, if the functional web is distributed over distant cortical areas, for instance, certain frontal and temporal areas, neurons in both areas should (i) share specific response features and (ii) show these response features only if the respective other area is intact.” (2002: 26, see also 27)

Pulvermüller’s line of reasoning 1. “If neurons in the functional web are strongly linked, they should show similar response properties in neurophysiological experiments. 2. “If the neurons of the functional web are necessary for the optimal processing of the represented entity, lesion of a significant portion of the network neurons must impair the processing of this entity. This should be largely independent of where in the network the lesion occurs. 3. “Therefore, if the functional web is distributed over distant cortical areas, for instance, certain frontal and temporal areas, neurons in both areas should (i) share specific response features and (ii) show these response features only if the respective other area is intact.” (2002: 26, see also 27)

A Memory Experiment (2002: 26-27)  Performed with macaque monkeys  Delayed matching – monkey must remember Monkey must keep in mind the shape or color of an object and perform a matching response after delay of several seconds  Neural activity detected in frontal and temporal lobes  Temporary lesion of frontal or temporal area leads to impaired stimulus specificity in other area

Reasoning from memory experiment  Temporary lesion of frontal or temporal area leads to impaired stimulus specificity in other area  “Together, these data provide evidence that neurons in both temporal and frontal areas (a) showed the same specific response features and (b) showed these response features if and only if the respective other area was intact…”  Compares language impairment vis-à-vis Wernicke’s and Broca’s areas (2002: 28) Not so fast! … the same specific response features?

Elsewhere he writes “similar” “If neurons in the functional web are strongly linked, they should show similar response properties in neurophysiological experiments.” (2002:26) N.B.: similar – not same!  Similar: Sharing some features There may be differences with respect to other features

Pulvermüller’s reasoning (cont’d) “These results obtained in memory experiments with macaque monkeys are reminiscent of well- known facts from … investigation into acquired language disorders …. These … studies … showed that prefrontal and temporal areas are most crucial for language processing. They also showed that lesions in either area can lead to aphasia, which in the majority of cases include deficits in both language production … and perception ….” (2002: 28)

Pulvermüller on Wernicke’s aphasia “… patients with Wernicke’s aphasia have difficulty speaking…. These deficits are typical…and cannot be easily explained by assuming a selective lesion to a center devoted to language comprehension.” (2002: 36-37)

Pulvermüller’s hypothesis on phonological word forms Friedemann Pulvermüller, The Neuroscience of Language, 2002: 52 “The functional webs realiz- ing phonological word forms may be distributed over the perisylvian area of the dominant left hemishpere. Circles represent local neuron clusters and lines represent reciprocal connections between them.”

Basic and complex functions  Phonological recognition is a basic function Located in Wernicke’s area  Speaking is a complex function A cooperative effort of several areas, including Broca’s area and Wernicke’s area Phonological recognition is a necessary component of speaking Wernicke: “Primary functions alone can be referred to specific areas…. All processes which exceed these primary functions…are dependent on the fiber bundles, that is, association.” Aphasia Symptom Complex (1874)

Wernicke’s Area and Speaking  Phonological images guide speech production  Phonological recognition monitors production  Compare.. Painting without visual perception Playing a piano without auditory perception  Conclusion: Of course phonological recognition (i.e. Wernicke’s area) plays a role in speech production

Paraphrasing Pulvermüller Altered quote: …patients with damage to the occipital lobe have difficulty drawing pictures…. These deficits are typical…and cannot be easily explained by assuming a selective lesion to a center devoted to visual perception. …patients with Wernicke’s aphasia have difficulty speaking…. These deficits are typical…and cannot be easily explained by assuming a selective lesion to a center devoted to language comprehension. The Neuroscience of Language (2002)

Re-examining the monkey memory experiment  Compare short-term verbal memory Hypothesis: reverberating activation between Broca’s area and Wernicke’s area If one of those areas is impaired, the reverberating activity is disrupted, leading to diminished activity in the other area  Same principle could apply in memory test in macaque monkey Reverberation between temporal lobe (recognition zone) and frontal lobe (action zone)  Does not require that the two areas share “same specific response features”

Conclusion: The components of a functional web are diverse  The phonological representation of a word may be seen as a functional web in the perisylvian area  But each component of the web has its own specific local function within that representation For example, phonological recognition in Wernicke’s area  If they are all the same, why have many of them, spread out over different areas? Compare Hypothesis III: Nodal specificity (below)

Elsewhere, Pulvermüller gets it right “…activation of the web, so to speak, completes itself as a result of the strong web-internal links. If the web of neurons is considered a memory representation of an object and each neuron to represent one particular feature of this object memory, the full ignition would be the neuronal correlate of the activation of the stored object representation. Such full activation of the object memory could occur if only a fraction of the features of the object are present in the actual input.” (2002: 29)

Why do the nodes in a web appear to have similar response features?  Not because each node has – on its own – response features similar to those of other nodes in the web  Simply because all the nodes are “tied together” in the web Therefore, all respond when the whole web is ignited  Actually they have, individually, very different response features E.g. in Wernicke’s area and Broca’s area

Reverberation in functional webs  Reverberation among connections in an established web strengthens activation  Experimental verification: Compare words and pseudo-words  Pseudo-words: phonologically OK but no meaning Real words show greater activation “About one-half second after the onset of spoken one-syllable words, high-frequency brain responses were significantly stronger compared to the same interval following pseudo-words.” (Pulverműller: 53)

Another word : pseudo-word experiment (Pulverműller 2002: 54-56)  Finnish pakko ‘compulsion’ takko : a pseudo word Same 2 nd syllable  Measurements used: MMN and MMNm (MMN : mismatch negativity) Larger for –ko of real word  Strongest difference at 200 ms  Subjects were watching a silent movie I.e., not paying attention

Finnish ‘pakko’ experiment: discussion  [-ko] produces activation in either context, since it is a syllable occurring in Finnish  Stronger activation in pakko pakko is an established word in Finnish That means it has established connections Established connections provide stronger activation

Hypothesis III: Nodal Specificity in functional webs  Every node in a functional web has a specific function  The nodes in each area of a functional web Constitute a subweb Their function fits the portion of cortex in which they are located  For example, Phonological recognition in Wernicke’s area Visual subweb in occipital and lower temporal lobe Tactile subweb in parietal lobe Each node of a subweb also has a specific function within that of the subweb WORDS IN THE BRAIN

Support for Nodal Specificity: the paw area of a cat’s cortex Column (node) represents specific location on paw

Support for Nodal Specificity: Columns for orientation of lines (visual cortex) Microelectrode penetrations K. Obermayer & G.G. Blasdell, 1993

Support for Nodal Specificity: Map of auditory areas in a cat’s cortex AAF – Anterior auditory field A1 – Primary auditory field PAF – Posterior auditory field VPAF – Ventral posterior auditory field A1

Hypothesis III(a): Adjacency  Nodes of related function are in adjacent locations More closely related function, more closely adjacent  Examples: Adjacent locations on cat’s paw represented by adjacent cortical locations Similar line orientations represented by adjacent cortical locations WORDS IN THE BRAIN

Support for Nodal adjacency: the paw area of a cat’s cortex Adjacent column in cortex for adjacent location on paw

Hypothesis III(b)  The nodes in each area of a functional web Constitute a subweb Each node of a subweb has a specific function within that of the subweb (Hypothesis III)  Functional specificity of subwebs: Each subweb has specific function within the web  Fits its location in the cortex  For example, Visual subweb in occipital and lower temporal lobe Tactile subweb in parietal lobe

A phonological subweb: /bil/ bil bi- -il Cardinal node for bill Subweb for bill

An activated functional web showing two subwebs V PR PA M C PP T Control of articulation Visual features

Hypothesis IV: Extrapolation to Humans  Hypothesis: The findings about cortical structure and function from experiments on cats, monkeys, and rats can be extrapolated to human cortical structure and function  In fact, this hypothesis is simply assumed to be valid by neuroscientists  Why? We know from neuroanatomy that, locally, Cortical structure is relatively uniform across mammals Cortical function is relatively uniform across mammals WORDS IN THE BRAIN

Hypothesis IV(a): Linguistic and conceptual structure  Hypothesis IV(a): The extrapolation can be extended to linguistic and conceptual structures and functions  Why? Local uniformity of cortical structure and function across all human cortical areas except for primary areas  Primary visual and primary auditory are known to have specialized structures, across mammals  Higher level areas are – locally – highly uniform WORDS IN THE BRAIN

Objection  Cats and monkeys don’t have language  Therefore language must have unique properties of its structural representation in the cortex  Answer: Yes, language is different, but The differences are a consequence not of different (local) structure but differences of connectivity The network does not have different kinds of structure for different kinds of information  Rather, different connectivities

Hypothesis V: Hierarchy in functional webs  A functional web is hierarchically organized Bottom levels in primary areas Lower levels closer to primary areas Higher (more abstract) levels in  Associative areas – e.g., angular gyrus  Executive areas – prefrontal  These higher areas are much larger in humans than in other mammals  Hypothesis V(a): Each subweb is likewise hierarchically organized WORDS IN THE BRAIN

Hierarchy in a visual subweb FORK Etc. etc. (many layers) A network of visual features V

Properties of Hierachy  Relates to general hierarchy in the cortex  Each level has fewer nodes than lower levels, more than higher levels Compare the organization of management of a corporation  Top level has just one node Compare the “CEO” The “C” node of a word web  Cardinal node Hypothesis VI

Hypothesis VI: Cardinal nodes  Every functional web has a cardinal node At the top of the entire functional web Unique to that concept For example, C /cat/ at “top” of the web for CAT  Hypothesis VI(a): Each subweb likewise has a cardinal node  At the top level of the subweb  Unique to that subweb  For example, V /cat/ At the top of the visual subweb WORDS IN THE BRAIN

Cardinal nodes of a functional web Some of the cortical structure relating to fork V M C T P PA PP Cardinal node of the whole web Cardinal node of the visual subweb Each node shown here is the cardinal node of a subweb

(Part of) the functional web for the concept CAT V P A M C The cardinal node for the entire functional web T Cardinal nodes of subwebs

The argument against cardinal nodes  Pulvermüller: “It is not necessary to assume a cardinal node” (p. 24)  Arguments by others (directed against “grandmother nodes): Not enough flexibility Not enough availability  Response to the arguments: The cardinal node of a hierarchical web is not a “grandmother node” as usually understood It is supported by the hierarchy principle Compare  CEO of a corporation  President of the U.S.

The “Grandmother Node” (Cardinal node for your grandmother)  Grandmother node A node that represents “grandmother”  An untenable hypothesis, according to the usual conception  But two separate conceptions to be distinguished A node that represents “grandmother” all by itself A node whose receptive field is “grandmother”

The untenable grandmother node  A node that would recognize grandmother all by itself Such a node would have to be extraordinarily complex  How could one node recognize grandmother In different positions/postures In different clothing At different ages Criticisms of such a conception are well-founded  Such a hypothesis involves local representation without distributed representation

A sophisticated grandmother node  GRANDMOTHER has a distributed representation  It also has a cardinal node (local representation) A ‘grandmother node’ in the sophisticated sense It represents a specific value: GRANDMOTHER Its receptive field is “grandmother”  It works because it is the cardinal node of an entire functional web Other nodes in the web handle  The details  A range of diverse perceptual properties

Arguments against ‘grandmother nodes’  They usually assume that the local representation is representing a concept (like ‘grandmother’) all by itself i.e., Local representation without distributed representation  i.e., without a supporting web

Arguments against local representation 1. Recognizing new things and producing motor responses to new things are problematic on the local-coding theory 2. The patterns recognized visually by a human in a lifetime vastly outstrip the number of sensory processing neurons in the entire human nervous system Churchland & Sejnowski The Computational Brain MIT Press, 1992, p. 163

Arguments against local representation 1. Recognizing new things and producing motor responses to new things are problematic on the local-coding theory 2. The patterns recognized visually by a human in a lifetime vastly outstrip the number of sensory processing neurons in the entire human nervous system These arguments are directed against the naïve conception of the grandmother node.

Arguments against local representation 1. Recognizing new things and producing motor responses to new things are problematic on the local-coding theory This argument assumes that such a node recognizing grandmother all by itself. But it is the whole functional web that recognizes grandmother. Each part of this web naturally responds to a wide range of values, including novel values.

Arguments against local representation 2. The patterns recognized visually by a human in a lifetime vastly outstrip the number of sensory processing neurons in the entire human nervous system Churchland & Sejnowski 1992:163 On the contrary, the web can accommodate recognition of multiple new exemplars without the need for recruiting additional nodes. Not a problem after all. New nodes are needed only for new learning.

Support for the cardinal node hypothesis 1. It follows from the hypotheses of nodal specificity and hierarchy A hierarchy must have a highest level The node at this level must have a specific function 2. It is needed for ignition of the whole web from activation of part of it For example, to activate the phonological representation from the visual 3. It is automatically recruited in learning anyway, according to the Hebbian learning hypothesis

More support for cardinal nodes  The distributed network as a whole represents the concept (e.g. FORK )  The whole can evidently be activated by any part of the network From seeing a fork From eating with a fork Etc.  The cardinal node provides the coordinated organization that makes such reactivation possible

Reactivating the functional web  When the cardinal node (the integrating node) is activated, it can activate the whole (distributed) functional web Without it, how would that be possible? E.g., activating conceptual and perceptual properties of cat upon hearing the word cat From phonological recognition to concepts From visual image to phonological representation

Cardinal nodes and the linguistic sign  Connection of conceptual to phonological representation  Consider two possibilities 1. A cardinal node for the concept connected to a cardinal node for the phonological image 2. No cardinal nodes: multiple connections between concept representation and phonological image supported by Pulvermüller (2002)

Pulvermüller’s hypothesis: No cardinal nodes Friedemann Pulvermüller, The Neuroscience of Language, 2002 Phonological representation: a distributed representation in the perisylvian area Meaning of a verb Meaning of a visual object

Functional Webs acc. to Pulvermüller 1.Distributed representation of form and of meaning This part is correct 2.Multiple connections between form and meaning Runs counter to the linguistic evidence Implication: parts of the phonological representation connect to parts of the meaning Example: walk - WALK  [w-] or [-k] for action with legs?

Implications of possibility 2  No cardinal nodes: multiple connections between concept representation and phonological image  I.e., different parts of meaning connected to different parts of phonological image  Consider fork Maybe /f-/ connects to the shape? Maybe /-or-/ connects to the feeling of holding a fork in the hand? Maybe /-k/ connects to the knowledge that fork is related to knife?  Conclusion: Possibility 2 must be rejected

Support for the cardinal node hypothesis – 3  It is automatically recruited in learning according to the Hebbian learning principle  Even if it weren’t there it would soon be recruited as a result of co-activation of its linked properties  This is the operating principle for building a functional web from bottom up At each level, co-occurring properties will activate a node at next higher level  That newly activated node represents the combination of those properties This process continues up to top of hierarchy

end