1. Towards Automatic Fluency Assessment
Suma Bhat

2. The Problem Fluency – subjective quantity
Design of right quantifiers critical
Measurement of the quantifiers required
Problem: Automatic Assessment

3. Problem is Hard Fluency Subjective quantity Not readily measurable
More than just the opposite of disfluency
Key: design of good quantifiers

4. Previous Work Main work by Cucchiarini et al.
Quantitative assessment of fluency in speech possible
Measure quantifiers of syllable rate and frequency of pauses
Use of ASR

5. Low level acoustic variables are good quantifiers of fluency
Our Work
Previous work used language specific training data to build ASR
Our focus:
low-level acoustic measurements
Our thesis:
Low level acoustic variables are good quantifiers of fluency

6. Experiments Start with speech signal
View acoustic data at a coarse level
Syllables well represented by corresponding vocalic nuclei
Vocalic nuclei and silent pauses can be detected automatically
Goal: correlate with human judgment

7. Data Data from rated assessment
Classroom recording of 2nd language Chinese speech
20s snippets rated for
Speech Flow
Phonological Control
Lexical Accuracy
Disfluency
Delivery Skills
Fluency
Fluency most correlated with Speech Flow,Disfluency

8. Assessment of Fluency Speech Flow Disfluency
dur1=duration of speech without silent pauses
dur2= total duration of speech
Name
Definition
Articulation Rate
Number of syllable-nuclei/dur1
Rate of Speech
Number of syllable nuclei/dur2
Phonation/time ratio
dur1/dur2
Mean length of silent pauses
Mean length of silent pauses
Number of silent pauses per second
Number of silent pauses/dur2
Number of filled pauses per second
Number of filled pauses/dur2
Speech Flow
Disfluency

9. Acoustic Measurements
Downsample to 16K
Use intensity information
Segment utterance into regions of speech and silence
silent pause related information
Detect vocalic regions in the speech segments
syllable related information

10. Correlation with Human Rating
Acoustic Measurement
Human Rated Speech Flow
Human Rated Disfluency
Articulation rate
0.19
0.38
Rate of speech
0.30
0.5
Phonation time ratio
0.58
0.72
Length of silent pause
-0.36
-0.35
Frequency of silent pause
-0.4
-0.52
Frequency of Filled pause
-0.21
-0.43

11. Mean Values of Quantifiers
Acoustic Measurement
Non-fluent
Fluent
Articulation rate (/s)
2.59
3.21
Rate of speech (/s)
1.22
1.93
Phonation time ratio
0.47
0.59
Length of silent pause (s)
1.18
1.05
Frequency of silent pause (/s)
0.83
0.58
Frequency of Filled pause (/s)
0.1
0.16

12. Conclusion Key Acoustic Features
Phonation Time ratio most positively correlated with Speech Flow
Frequency of silent pauses most negatively correlated with Speech Flow
Summary:
Quantifiers obtained by low-level acoustic measurements useful for non-rated assessment

13. Looking Ahead Measurements on more rated speech
Detection of filled pause
Look for additional quantifiers
Poor pronunciation
vocabulary richness
Automatic classification of fluency