1. Brain Function I: Evidence from Linguistics
Brain, Mind, and Belief: The Quest for Truth
Brain Function I: Evidence from Linguistics
The Neurological Basis
of Language and Thought
Language, for all its seeming complexity, is more amenable to analysis than other cognitive structures, so that investigation of language is one of the best ways to proceed to an understanding of human mind and nature.
Tim Pulju, 1990

2. Agenda for Today The Problem: How does the brain work?
History of the study of brain function
Common errors to be avoided
Tool-driven inquiry
The misapplied metaphor
Help from the study of linguistic structure
Claim: As language works in the brain, so the brain works in general
Therefore, If we can understand how language works, we will know how the brain works
Relational networks
Cortical columns of neurons

3. History of the study of brain function I
Early investigators in the 19th century came up with the idea of locationism: Local areas of the cortex have specific functions
Franz Joseph Gall ( )
promoted the idea
His followers took it too far
As a result, the idea of
locationism was widely
discredited

4. History of the study of brain function II
Locationism: Local areas of the cortex have specific functions
Paul Pierre Broca ( )
Stroke patient with impaired speech
Autopsy after patient’s death
Damage in lower left frontal lobe
Area now known as Broca’s area
Responsible for articulation of speech

5. History of the study of brain function III
The origin of Connectionism
Like locationism but more sophisticated: Local areas of the cortex have highly specific simple functions, and complex functions are carried out by multiple interconnected areas
Carl Wernicke ( )
Stroke patient unable to comprehend speech
Damage in upper left temporal lobe
Area now known as Wernicke’s area
Responsible for speech comprehension

6. Connectionism Connectionism includes a version of locationism
But is more sophisticated
Each local area performs a very specific simple function
Complex functions require multiple local areas acting together
They can act together because they are interconnected
CONNECTIONS RULE!

7. Two basic language areas
Primary Somato-
sensory Area
Leg
Primary
Motor Area
Trunk
Arm
Hand
Wernicke’s
area
Fingers
Mouth
Phonological
Production
Phonological
Recognition
Broca’s area
Primary Auditory
Area
Primary
Visual Area

8. Arcuate Fasciculus (from langbrain website)
Connects Wernicke’s area to Broca’s area

9. History of the study of brain function IV
The decline and revival of Connectionism
As with Gall, followers of Wernicke were too speculative and went too far, and the whole idea was discredited for several decades
Finally revived in the 1960’s by Norman Geschwind ( )
Now widely accepted by neurologists (but criticized by some psychologists)

10. History of the study of brain function V
Two major methods of investigation
Lesion studies
E.g., Broca and Wernicke and Geschwind
If area A is damaged and function F is impaired, then A must have function F (or at least contribute to F)
Functional Brain Imaging
A recent innovation
Made possible by technological advances
Now very widely used
Location-based but sometimes without the sophistication of connectionism

11. Functional Brain Imaging
Electro-Encephalography (EEG)
Excellent temporal resolution
Very poor spatial resolution
Positron Emission Tomography (PET)
Poor spatial resolution
Very poor temporal resolution
Functional Magnetic Resonance Imaging (fMRI)
Spatial resolution better than PET
Temporal resolution a little better than PET
Magneto-Encephalography (MEG)
Spatial resolution not so good

12. Positron emission tomography (PET)
PET shows areas of increased cortical metabolism
Spatial resolution: 5-10 mm
How good is that?
Under one sq mm of cortical surface, 130,000 neurons
Temporal resolution: “…on the order of minutes…”
(A. Papanicolaou, Fundamentals of Functional Brain Imaging (1998), p. 14)

13. Functional Magnetic Resonance Imaging (fMRI)
Measures the amount of oxygenated blood supplied to different areas of the brain
Common abbreviation: rCBF (regional cerebral blood flow)
When a group of neurons increases its signaling rate, its metabolic rate increases

14. An fMRI example Areas of the brain used in working memory
SITE/ARTICLES/love.asp

15. Properties of fMRI Temporal resolution: Not very good
Image reflects the increase in oxygenated blood that occurs 5 to 8 seconds after the neurons fire
Spatial resolution:
Better than PET
But it is unclear whether the imaged area is precisely the area involved in the activity
The flow of oxygenated blood into the depleted area may also flow into neighboring vessels in areas where neural firing did not occur

16. Magnetoencephalography (MEG)
MEG (MagnetoEncephaloGraphy) measures the magnetic field around the head
Magnetoencephalography
magnetic
brain
picture
production of

17. How MEG works
An electric current is always accompanied by a magnetic field perpendicular to its direction
MEG records the magnetic flux or the magnetic fields that arise from electric currents in neurons
Magnetic flux lines are not distorted as they pass through the brain tissue because biological tissues offer practically no resistance to them
Therefore, MEG is more accurate than EEG

18. Magnetic flux from source currents
Magnetometer
Source current

19. Recording of Magnetic Signals

20. Temporal Resolution of MEG
Excellent – unlike fMRI and PET
Therefore, it is possible to discern the temporal order of activation of cortical areas
MEG has potential to detect the activation of several brain regions as they become active from moment to moment during a complex function such as recognition

21. A major challenge of MEG
The cortex is a parallel processor
Hundreds or thousands of dipoles can be active simultaneously
Multiple dipoles make comprehensive inverse dipole modeling virtually impossible
Hence, compromises are necessary
Sample larger time spans (up to 500 ms)
Sample larger areas (up to several sq cm)

22. Some MEG results: Speech recognition
Hemispheric Asymmetry
Wernicke's Area

23. Wernicke’s area in Spanish-English bilinguals
From MEG lab, UT Houston

24. Spatial Resolution How accurately is location determined? EEG: Poor
PET: Fair – 4-5 mm
fMRI: Fair – 4-5 mm
MEG: Fairly good – 3-4 mm or less
Under good conditions
How good are these figures?
under 1 sq mm of cortical surface,140,000 neurons

25. Temporal resolution Temporal resolution
Key neural events can occur within 5 ms
Terrible: PET
40 seconds and up
Pretty bad: fMRI
10 seconds or more
Excellent: MEG and EEG
Instantaneous
Theoretically it is possible to do ms by ms tracking, to follow time course of activation
But such tracking is usually difficult or impossible
The inverse problem
Too many dipoles at each point in time

26. Sensitivity of Imaging Methods
All of the methods have limited sensitivity
MEG
10,000 dendrites in close proximity have to be active to detect signal
PET and fMRI
Similar limitations
Any activation that involves fewer numbers goes undetected

27. Faulty thinking in neuroscience I: Tool-driven inquiry
Tool-driven inquiry: letting the available tools shape the investigation
Like looking for the lost car keys under the street light instead of where they got lost
The available tools: Brain imaging machines
What they are good for: determining locations of brain activity
Therefore, what do they investigate?
Locations of brain activity
The question being asked: Where?
The questions not being asked: What?, How?
What is going on?
How does the brain work?

28. More on the lack of interest in what/how
There are no available machines for investigating the what/how question
Experiments have not been devised for investigating the what/how question
It is necessary to rely on thinking
Scientists believe that doing science is conducting experiments and using high-tech machinery
Thinking is done by theoreticians
Akin to philosophers and poets

29. A mitigating circumstance?
Many have not realized that the what/how question is important
They may assume it is already known:
The brain is assumed to work like a computer
A symbol-manipulating device
This assumption is unwarranted

30. Faulty thinking in neuroscience II: The misapplied metaphor
The brain is assumed to work like a computer
This assumption is unwarranted
The computer is a symbol-manipulating device
Not a connectionist system
An example of faulty thinking:
The misapplied metaphor
The brain works by means of connectivity and operations upon its connectivity

31. The Cortex is a Network Entirely different structure than that of computers
Connectivity as key property of brain structure
Symbol-manipulation is the key property of computers
The cortex operates by means of connections
Transmission of activation along neural pathways
Changes in connection strengths

32. Computers and Brains: Different Structures, Different Skills
Flexible, fault tolerant
Slow processing
Association
Intuition
Adaptability, plasticity
Self-driven activity
Unpredictable
Self-driven learning
Computers
Exact, literal
Rapid calculation
Rapid sorting
Rapid searching
Faultless memory
Do what they are told
Predictable

33. Things that brains but not computers can do
Acquire information to varying degrees
“Entrenchment”
How does it work?
Variable connection strength
Connections get stronger with repeated use
Perform at varying skill levels
Degrees of alertness, attentiveness
Variation in reaction time
Mechanisms:
Global neurotransmitters
Variation in blood flow
Variation in available nutrients
Presence or absence of fatigue
Presence or absence of intoxication

34. How to study the what/how question
You have to think harder
Also, we can make use of findings from structural linguistics
Language as the key to unlock mysteries of the mind
If cortical structures for language are like those for other high-level skills
Then if we figure out language, we also have the answer to how other high-level intellectual processing works

35. Thinking harder Avoid metaphorical thinking
The brain is not a computer
Not like a human being with paper & pencil & books
In fact it is not like anything else
It is itself: the brain

36. How does your brain tell your fingers what to do?
Question to daughter (age 6):
How does your brain tell your finger to hit that key on the piano?
Sarah:
Well my brain writes a little note and sends it down to my finger, …
What really happens?
Neurons in the motor cortex send activation (nerve impulses) down (through subcortical structures and the spinal cord) to neurons that activate the muscles that make my finger move

37. Auditory Imagery Auditory images of words, music, etc.
We can hear things in our heads
What is an auditory image?
What does it consist of?
Sound?
There is no air inside the head to vibrate
What hears it?
There are no little ears inside the head

38. Visual Imagery Visual images of people, buildings, etc.
What is a visual image?
What does it consist of?
Is it a little picture?
If so, where are the eyes to see it?
What is it drawn on?
Where is the visual perception system to interpret it?
If not, what?

39. Vision When you see something.. A picture on your retina?
Something in your brain looks at it?
Are there a couple of little eyes inside?
A picture somewhere inside your brain?
Same problems: no eyes inside
And if there were, they would have to be supported by a visual perception system

40. Compare the TV set Does it have little people inside?
Similarly, no pictures inside the brain
No sounds inside the brain
No words or other symbols inside the brain

41. How vision really works
There is no picture on the retina, just neurons that get activated by light
Some of them get activated by color
The visual perception system learns during early childhood to integrate configurations of these little dots into larger units
Next higher level: larger configurations
Many levels up, recognition of objects
The brain goes through a long process of learning, to build these many levels
More on this next week!

42. The Nature of Language Some history Louis Hjelmslev
Prolegomena to a Theory of Language (1943/60)
Linguistic structure is a system of relationships
”The postulation of objects as something different from the terms of relationships is a superfluous axiom and con-sequently a metaphysical hypothesis from which linguistic science will have to be freed.”

43. Understanding linguistic units as purely relational I
dog
d - o - g
Symbols?
Objects?

44. Understanding linguistic units as purely relational II
Seems to be one unit
Three phonemes (or graphemes), in sequence
dog
d o g

45. Understanding linguistic units as purely relational III
Grammatical properties
DOG Noun
The meaning of dog – a concept
dog
The object we are considering
d o g

46. Understanding linguistic units as purely relational IV
DOG Noun
dog
d o g

47. Understanding linguistic units as purely relational V
DOG Noun
We can remove the symbol with no loss of information.
Therefore, it is a connection, not an object
d o g

48. Another way of looking at it
DOG Noun
dog
d o g

49. Another way of looking at it
DOG Noun
d o g

50. The phonological (or graphemic) segments
DOG Noun
What about these segments? Are they objects?
d o g

51. The phonological (or graphemic) segments
What about these segments? Are they objects?
d o g
/d/ as a phoneme has components:
Articulation: closure of mouth (so also /g/)
Position: Tongue tip against alveolar ridge
Voiced (compare /t/)
(Also auditory components: more complex)

52. A closer look at the segments
boy
(toy)
(Bob) b o y
The phonological
segments also
are just locations
in the network –
not objects
Phonological
features

53. Relations all the way
Perhaps all of linguistic structure is relational
It’s not relationships among linguistic items; it is relations to other relations to other relations, all the way to the top – at one end – and to the bottom – at the other
In that case the linguistic system is a network of interconnected nodes

54. What is at the bottom?
In the system of the speaker, we have relational network structure all the way down to the points at which muscles of the speech-producing mechanism are activated
At that interface we leave the purely relational system and send activation to a different kind of physical system
For the hearer, the bottom is the cochlea, which receives activation from the sound waves of the speech hitting the ear

55. What is at the top?
Somehow at the top there must be meaning

56. What are meanings? In the Mind For example, DOG The World Outside DOGC
DOG
Perceptual
properties
of dogs
All those dogs
out there and
their properties

57. What are meanings? In the Mind For example, DOG The World Outside
Conceptual
properties
of dogs
Perceptual
properties
of dogs
All those dogs
out there and
their properties

58. What are meanings? In the Mind For example, DOG The World Outside
Conceptual
properties
of dogs
Perceptual
properties
of dogs
All those dogs
out there and
their properties
Also relational network structure

59. The concept DOG We know what a dog looks like
A visual subnetwork, in occipital lobe
We know what its bark sounds like
An auditory subnetwork, in temporal lobe
We know what its fur feels like
A somatosensory subnetwork, in parietal lobe
All of the above..
constitute perceptual information
are subnetworks with many nodes each
Are interconnected into a larger network

60. The concept of DOG as a network
A – Auditory
C – Conceptual
M – Memories
P – Phonological
T – Tactile
V - Visual T P A C V M
Each node in this diagram
connects to a subnetwork
of properties

61. Objects in the mind?
When the relationships are fully identified, the objects as such disappear, as they have no existence apart from those relationships
“The postulation of objects as some-
thing different from the terms of
relationships is a superfluous axiom
and consequently a metaphysical
hypothesis from which linguistic
science will have to be freed.”
Louis Hjelmslev (1943/61)

62. How the mind operates Example: language
People are able to use their languages
Speaking
Writing
Comprehension
Such operation takes the form of activation of lines and nodes
Activation travels from node to node
Along connecting lines
Compare the highway system
A network
Operation: vehicles move along the roads

63. Two different network notationsab
Abstract notation
Bidirectional
a b
Upward
Downward ab b
a b f
a b
Narrow notation

64. Narrow relational network notation
Represents network structures in greater detail
internal structures of the lines and nodes of the more abstract notation
Closer to neurological structure
Each node represents a bundle of neurons
Links represent neural fibers (or bundles of fibers)

65. Abstract and narrow network notation
The lines and nodes of the abstract notation are abbreviations for more complex structures
Compare the representation of a divided highway on a highway map
In a more abstract notation it is shown as a single line
In a narrow notation it is shown as two parallel lines of opposite direction

66. AND vs. OR 27
AND OR
twenty seven 12
twelve dozen

67. AND vs. OR: Internal Structure (narrow notation)2 1

68. Thresholds in Narrow Notation
OR AND 1 2 3 4
– You can have intermediate
degrees, between AND and OR
– The AND/OR distinction was a
simplification anyway —
doesn’t always work!

69. Levels of precision: Add another
Abstract relational network notation
Narrow relational network notation
Neural structures

70. Narrow RN notation and neural structures
Question: Are relational networks related in any way to neural networks?
We can find out
Relational networks were devised as a means of accounting for linguistic structure
Their properties depend on properties of language
Evidence for them comes from language, not from the brain
Narrow RN notation can be viewed as a set of hypotheses about brain structure and function
Properties of narrow RN notation can be tested for neurological plausibility

71. Some properties of narrow RN notation
Lines have direction (they are one-way)
But they tend to come in pairs of opposite direction (“upward” and “downward”)
Connections are either excitatory or inhibitory
Nerve fibers carry activation in just one direction
Cortico-cortical connections are generally reciprocal
Connections are either excitatory or inhibitory (from different types of neurons, with two different neurotransmitters)

72. More properties as hypotheses
Neurons have different thresholds of activation
Inhibitory connections are of two kinds
(Type 2: “axo-axonal”)
All are verified
Nodes have differing thresholds of activation
Inhibitory connections are of two kinds
Additional properties – (too technical for this presentation)
Type 1
Type 2

73. The node of narrow RN notation vis-à-vis neural structures
The node (of narrow RN notation) corresponds (not to a single neuron but) to a bundle of neurons
The cortical column
A column consists of neurons stacked on top of one another
More on this next week!

74. T h a t ‘ s i t f o r t o d a y !