Stress as Attention Modulation and the Roots of its Formal Properties Kenneth de Jong Indiana University Jones.ling.indiana.edu/~kdejong & (slides at jones.ling.indiana.edu/~kdejong/Buffalo_stress.ppt)

Stress as Attention Modulation and the Roots of its Formal Properties Work reported here in collaboration with Byung- jin Lim, Kyoko Nagao, Eric Oglesbee, Noah Silbert, and Bushra Zawaydeh, and supported by NSF: BCS & BCS , and NIDCD: R03 DC04095

Overall Thesis The formal properties of linguistic stress can be explained terms of... the definition of linguistic stress –Linguistic stress = conventionalized attention modulation at the level of the syllable. how stress begins historically –Attention modulation 'grows from' seeds of syllable-level prominence the function of linguistic stress –Attention modulation is one strategy for governing focus on informationally important units

Types of Explanations (deJong, 2007) Physiological facts and constraints on control Hardware Motoric propensities which encourage certain articulatory coordinations Activeware OR … Requirements of communication with a listener Shareware OR … It’s the way mom & dad did it Ancestorware

Types of Explanations (deJong, 2007) Need to consider interaction of activeware & shareware factors As they are encoded in the ongoing aggregation of a language … ancestorware

Overall Thesis The formal properties of linguistic stress can be explained terms of... the definition of linguistic stress –Linguistic stress = conventionalized attention modulation at the level of the syllable. how stress begins historically –Attention modulation 'grows from' seeds of syllable-level prominence

Overall Thesis The formal properties of linguistic stress can be explained terms of... the definition of linguistic stress Shareware –Linguistic stress = conventionalized attention modulation at the level of the syllable. how stress begins historically –Attention modulation 'grows from' seeds of syllable-level prominence

Lindblom’s H r + H o Model (Lindblom 1990) Hyperarticulation (Hr) - Hypoarticulation (Ho) Hr = output oriented constraints (Shareware, since these constraints are those imposed by the function of matching speaker and listener) Ho = production-oriented factors (Hardware & Activeware, since these factors are aspects of speakers’ motor control systems) Not OR …; but AND

Lindblom (1990) “ For the ideal speaker, H+H claims that … adaptations reflect his tacit awareness of the listener’s access to sources of information independent of the signal and his judgement of the short-term demands for explicit signal information.”

Evidence behind H r + H o Ho: motor optimization –Lindblom’s work over a long period of time shows economization of articulatory actions (e.g. tip & jaw movement), and ‘soft’ temporal effects on undershoot. –Economization, if not omnipotent, appears omnipresent Hr: intelligibility boost – The best speech intelligibility enhancement device is the speaker –Speakers know how get themselves heard

Extension of H r +H o (de Jong, 1991; 1995; 1998) Phonetic variation due to linguistic stress looks like Lindblom’s Hyperarticulation. Specified not in terms of specific features, but in terms of a functional increase in intelligibility At that time: This was Uhglee. Essentially stress is open-ended adaptation, introducing the unanswered question of how to characterize intelligibility

Stress functionality Productions of stressed items are given more attention (de Jong, 1991; 1995;1998) Perceptions of stressed items are more important for intelligibility (Cole, et al, 1978, Cole and Jakimik, 1980; Bond and Garnes, 1980; Terken and Nooteboom, 1987) A stress system = a convention in which both speakers and listeners dynamically pay more attention to certain syllables than to others

Shareware Explanation - Attentional Dynamics Auditory attention work by Mari Jones & others (Jones, 1976; Jones & Boltz, 1989; Jones & Yee, 1993; Large & Jones, 1997) –attentional selectivity –attentional capture –attentional integration –temporal expectancy

Stress variation - Attentional dynamics Part 1. attentional selectivity: some parts of a stimulus are more readily acted upon than others Stress => some syllables are attentionally selected

Stress variation - Attentional dynamics Part 2. attentional selectivity arises from ---> attentional capture: parts which suddenly change in salient dimensions tend to garner such selective advantages Attention is attracted to acoustic events where sudden changes take place --> syllables are typically characterized by a sudden onset of acoustic energy Attention is attracted to acoustic events where sudden changes take place, e.g. pitch accents --> stressed syllables often act as sites for pitch accent docking

Stress variation - Attentional dynamics Part 3. attentional selectivity also ---> may exhibit attentional integration: aspects which get attended to as a unit are those which work together to define an object or event works on portions of speech which cohere in audition, such as syllabic groupings Hence, again, stress is a property of the syllable (MORE on this later on, though)

Stress variation - Attentional dynamics Part 4. Attentional modulation is generally governed by temporal expectancy. High attention areas may, under appropriate conditions, come at temporally predictable intervals. Hence stress is sometimes governed by rhythmic, metrical patterning.

Stress variation - Attentional dynamics What we have is a Shareware explanation for stress-related phonetic variability.

Focus & Stress (de Jong & Zawaydeh, 2002; de Jong, 2004) To firm up link between stress & hyperarticulation … have speakers explicitly do hyperarticulation, contrasting a lexical target with another one. - two types: –'broad' lexically contrastive focus (e.g. 'bad', not 'good') –'narrow' phonologically contrastive focus (e.g. 'bad', not 'bat') (c.f. van Heuven, 1994)

Vowel Duration & Voicing Stress by voicingFocus by voicing

Type of Focus - example results Stress & focus have similar effects Focus interacts with stress Explicit phonological focus is less consistent across speakers

English vs. Arabic Voicing- stress interaction effects are different

Arabic vs. English quantity Stress degree is different in the two English: long & short vowel differences get larger with focus and stress in combination Arabic: no interaction between stress & focus

Interim Summary Contrastive dimensions are expanded by stress and focus Dimensions that are not used are not expanded, e.g. voicing -/-> vowel duration in Arabic English has more of a stress system than Arabic; focus effects are strongly localized in stressed positions in English

Next Survey what effects are found for stress and focus -> tells us more about relationship between stress and syllables

Focus & Stress Effects -Connect contrastive dimensions to focus modulations: onset voicing and VOT & F0 in voicing contrasts (Oglesbee, 2008) -VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

Focus & Stress Effects -Connect contrastive dimensions to focus modulations: onset voicing and VOT & F0 in voicing contrasts (Oglesbee 2008) -VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

Oglesbee (2008) Measured perceptual sensitivities in a goodness estimation task to VOT, F0 contour, and intensity dynamics for /p/ and /b/ Compared productions of /p/ and /b/ in focus and non-focus positions Subjects created larger differences with focus in dimensions with perceptual sensitivity Dimensions: –All: VOT –S1: vowel amplitude ramp (/b/ sharper) –S2: F0 contour (/p/ higher)

Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

Non-high front vowels -> Vowel quality Stress by voicing Dashed arrows indicate voicing effects Solid arrows indicate effects going from: Unstressed -> secondary stressed Secondary -> primary stressed - F2 (Hz) - F1 (Hz)

Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

Silbert & deJong (JASA, 2008) Examined English /f/ /v/ /s/ & /z/ Prevocalic, post-vocalic & focus, non-focus Measured intensity, duration, spectral quality and dynamics of quality Focus affects only duration and intensity -> NO spectral effects

Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre is not affected by focus -> rigid motor constraints - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre is not affected by focus -> rigid motor constraints - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

Voicing -> Vowel duration Stress by voicing Focus by voicing

Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre is not affected by focus -> rigid motor constraints - More CONSONANTS: other consonantal cues are heavily affected

Why Syllables? (2) Stress effects strongly localized in vocalic nucleii Consonant effects similarly most readily apparent in nucleii Consonant acoustics restricted to effects in terms of durational and intensity properties All of which suggests a model of syllable nucleii as motorically fluid repositories of variation Consonantal margins are motorically constrained

Summary 2 Stress - Hyperarticulation link is generally supported, BUT … Stress looks to be a better exemplar of Hyperarticulation than does focus Um, huh? Why? Stress is part of English conventions: it’s actually Ancestorware

Summary 3 Suggests a model with two components 1)Shareware: Hyperarticulation happens according to speakers’ perceptions of listeners’ ability to perceive -This hyperarticulation in general is unevenly distributed in the signal 2)Ancestorware: Hyperarticulation effects get contributed to the pool of experiences that the listener has with particular lexical items -Shareware factors get encoded in Ancestorware

Overall Thesis The formal properties of linguistic stress can be explained terms of... the definition of linguistic stress Shareware –Linguistic stress = conventionalized attention modulation at the level of the syllable. how stress begins historically –Attention modulation 'grows from' seeds of syllable-level prominence

Overall Thesis The formal properties of linguistic stress can be explained terms of... the definition of linguistic stress Shareware –Linguistic stress = conventionalized attention modulation at the level of the syllable. how stress begins historically Ancestorware –Attention modulation 'grows from' seeds of syllable-level prominence

Ancestorware component Some formal properties may be explained by historical language dynamics Quantity sensitivity in stress systems

Quantity sensitivity Stress location often is preferential as to what sort of syllable it resides on Formal property used to determine preference is weight (Newman, 1972) Weight is determined with respect to an implicational hierarchy of the following form: Long V > Coda son > Coda obstruents > Short vowels [..V:C] > [..V:] > [..VR] > [..VO] > [..V] Languages differ as to where boundary between light and heavy syllables falls

Quantity sensitivity problems Gordon (1999) –Surveyed 388 stress rules –Languages have different weight criteria for different phonological phenomena –Stress rules virtually always treat all consonants the same when determining weight Ahn (2000) –Surveyed 136 stress rules –Found huge differences in the frequency of different stress systems

Ahn (2000) In the 23 unbounded quantity sensitive system, consonants never count as heavy Only vowels get stress, if stress position is not tied to a word edge Quantity Insensitive Quantity Sensitive Bounded7045 Unbounded23

Ahn (2000) Further, foot type (iamb vs. trochee) is strongly linked to which word edge is important x [ ... x [ ... x...  ] x...  ] x...  ] Etc

Ahn (2000) Further effects –81% of languages: Heavy syllable stresses are one syllable closer to a word edge than light syllable stresses are –The effect of consonants in stress rules is almost entirely restricted to that of pushing the target vowels away from word edge

Korean Stress Utterance final or word initial (Polianov, 1936) Word initial (Huh, 1985; JS Lee, 1992) Unbounded, but if no long vowels then word initial quantity sensitive (HB Lee, 1974) Unbounded system, closest to initial edge (J Yu, 1989) Word initial, quantity sensitive (HY Lee, 1990; JK Kim, 1998) Phonemic (Zong, 1965; Cho, 1967) None of the above (SA Jun, 1993; me, 1994)

Stress vs. Non-stress Systems (Beckman, 1986) English: –Perceptual cues to stress location include large effects of duration, amplitude, vowel quality, but largely cued by interpretation of F0 as indicating a pitch accent Japanese: –Perception of prominence dominated by interpretation of F0 pattern –Stress not tied to syllables with associated accents

Korean non-stress No durational/intensity/spectral differences F0 determined by word initial rise (%LH) Lim (2001). The location of the H is stochastically weight sensitive; most H on second syllable, but if the second syllable is short, the H will sometimes appear on third syllable

Korean non-stress Potential historical dynamic - current

Korean non-stress Potential historical dynamic - soon?

Korean non-stress Potential historical dynamic

Quantity Sensitivity Results from historical accrual Syllables get picked out historically due to one of two properties –F0 peaks formerly associated with word edges –Phonemic vowel length These effects provide seeds for attention accrual

Overall Thesis The formal properties of linguistic stress can be explained terms of... the definition of linguistic stress Shareware –Linguistic stress = conventionalized attention modulation at the level of the syllable. how stress begins historically Ancestorware –Attention modulation 'grows from' seeds of syllable-level prominence

Why syllables? (again) Syllables might be implicated in both parts Shareware: - Syllables may act as integral units in the apportionment of attention. - Syllabic nucleii are ‘acoustic objects’ which exhibit attentional integration. Ancestorware: - Production variation is more readily and systematically encoded in vocalic nucleii, punctuated by less gradiently variable consonantal margins. - Focus variation along H+H lines gets localized in syllable nucleii, so over time, nucleii get stress. - Quantity sensitivity comes about (often) by the effect of boundary tones in attracting attention these syllabic nucleii