1. Multiple constraints in action
Sentence Processing:
Multiple constraints in action

2. Redundancy simplifies computation
We've seen how redundant cues of many kinds impact on word recognition
Today we will look at how such cues help us to decipher information in sentences so well
Why do we need redundant sources of information?
In information processing, redundancy removes ambiguity: as in the error-checking bits in computer communication
Uncertainty is decreased (probability of correct interpretation is increased) whenever something you know narrows down the range of what you don't know
Any regularity is by definition informational = increases probability of correctly predicting what you don't know (this is what it means to have a regularity)

3. Pattern and information
“Any aggregate of events or objects open (e.g., a sequence of phonemes, a painting, or a frog, or a culture) shall be said to contain ‘redundancy’ or ‘pattern’ if the aggregate can be divided in any way by a ‘slash mark’, such that an observer perceiving only what is on one side of the slash mark can guess, with better than random success, what is on the other side of the slash mark. We may say that what is on one side of the slash contains information or has meaning about what is on the other side. Or, in engineer’s language, the aggregate contains ‘redundancy’. Or, again, from the point of view of a cybernetic observer, the information available on one side of the slash will restrain (i.e, reduce the probability of) wrong guessing.”
Gregory Bateson
StepsTo An Ecology Of Mind
P. 104

4. It’s good to be on top!
Sentences benefit form being at the high-end of the linguistic hierarchy
Constraints from many levels help disambiguate sentences: syntactic, semantic, prosodic, pragmatic & ‘probabilistic’
They are all ‘probabilistic’ but the last category emphasizes that even if you know nothing at all about syntax, semantics, and pragmatics, language is still not random: some words appear more often than others, and some words are more likely follow certain words than others.
Recall Tom Landauer’s claim: 45% of sentences can be re-contructed from their words by maximizing co-occurrence probabilities

5. Order in probability
If language were entirely random, then all words would be equally likely, and we already know that:
a.) Some words are much more frequent than others
b.) The language system is sensitive to word probabilities- HF words are recognized more quickly than LF words
c.) Some words have a very narrow range of words that can follow them
i.e. articles are almost always followed by adjectives or nouns; never by verbs, articles, or prepositions

6. Order in probability
If language were entirely random, then all words would be equally likely, and we already know that:
d.) Syntactical constraints specify that words have a non-equal probability of appearing in certain places
i.e. subject usually before verb before object, or heads in leftmost position in VP and NP)
- But this is much enhanced by semantics: Compare ‘the N Ved into the X’ to ‘The train pulled into the X’
e.) Syntactical constraints specify that clauses have a non-equal probability of appearing in certain places
i.e.'if', 'in order to', 'because', 'since', 'whenever’ etc. all require closing clauses)

7. The psychological reality of syntax
Experimental evidence shows that words within a single clause are read more quickly than between-clause words
Readers are especially slowed when they reach an ambiguous section (garden path sentences like ‘The old man the boats’- we expect to discover what the old man did)
If you stop sentences midway and ask people to recall what they sentence just heard was, they tend to report by clause boundaries

8. The psychological reality of syntax
Other evidence also shows that clauses have psychological reality
If you play a click in the middle of auditory presentation of a sentence, subjects are much better at saying afterwards where it was if it comes between clauses than if it occurred within a clause
Moreover, they tend to report is as occurring at a clausal boundary far more often than it actually had occurred there.
This effect remains even if you remove accompanying disambiguating information such as pauses and changes in intonation that usually accompany clausal boundaries: i.e. in a robotic monotone voice clicks at clause boundaries were still better recalled than clicks within-clauses.
You get it even with (fake) subliminal clicks

9. Memory or language?
One question about these studies is whether is was a memory or language perception effect
If you ask people to indicate by pointing to a written sentence rather than by speaking, the effect was reduced, suggesting a memory effect
That is: suggesting that the effect may be due to chunking
But it is nevertheless still present- suggesting a language perception effect.

10. Memory or language?
Moreover…we can always m’u the distinction between language and memory
Following Chomsky, there may be a deep relation between how syntax chunks words into role and clauses, and how we think
Readers show pauses between clauses, and an effect of how many chunks there are to integrate (more = longer pauses)
Following Fauconnier, there may be a deep relation between how we carve up and chunk experience, and how that chunking is mapped onto our language system, so that memory chunking is linguistic (or reflects the same psychological constraints)
Even single words show amazing memory properties (Terry Deacon) - and then, so do narratives/stories/myths
Language is in part a technology for memory enhancement

11. Order in probability
If language were entirely random, then all words would be equally likely, and we already know that:
f.) Pragmatic rules limit what can come next to being related to what has already been communicated or what is currently happening.

12. Pragmatics: Grice’s Maxims
There are a great many complex pragmatic rules, but the most well-known are known as the Gricean Maxims:
i.) The Maxim Of Quality: Speakers should tell the truth as they know it, or explicitly acknowledge their uncertainty about the truth if they are aware of it
ii.) The Maxim of Manner: Speakers should strive to be clear, succinct, and unambiguous
iii.) The Maxim of Quantity: Speakers should say all that is necessary or required, but no more than that
iv.) The Maxim of Relation: Speakers should say only what is relevant

13. Order in probability
If language were entirely random, then all words would be equally likely, and we already know that:
g.) Semantics contributes to disambiguating uncertainty
The sentence 'The witness examined by the lawyer was useless' is read more slowly than the sentence 'The evidence examined by the lawyer was useless', even though both sentences have identical structure and phoneme count
The reason is that witnesses are active beings who can examine but evidence isn’t active and so cannot examine- it can only be examined
THE phrase 'The witness examined' is ambiguous at that point in a way that 'The evidence examined' is not.
We say the second sentence is semantically constrained in a way that the first is not.

14. Order in probability
If language were entirely random, then all words would be equally likely, and we already know that:
h.) We can (and we do) use prosody, stress, and pauses to disambiguate sentences such as these when they are spoken
i.e. speakers tend to automatically lengthen the final vowel in a word just before a clause boundary and also insert a pause

15. Context matters
Each one of these things contributes takes away a little bit of uncertainty in interpreting sentences
This can be shown experimentally: increased context makes words more likely to recognize under conditions of decreased exposure to that word or increased noisiness, and makes those words more likely to be recalled
Grosjean: words in context are recognized in ms of their onset (half their length); words out of context need over 100 ms more (average = 300 ms)
Cohort model applies to sentence processing too