The Computerised FDA Application Formulating A System of Acoustic Objective Measures for the Frenchay Dysarthria Assessment Tests

Background: The paper-based FDA  The type and severity of a given instance of dysarthria (disordered speech arising from impaired articulator control) is diagnosable by an assessment procedure known as the Frenchay Dysarthria Assessment (FDA) tests.  Two of the three FDA intelligibility tests are concerned with the measurement of intelligibility…but what exactly is intelligibility anyway? “ The degree of success in establishing communication between the sender and intended recipient of a message ”

Intelligibility, a very variable percept  Are both of these speech samples equally intelligible?  Initially, a listener will find it more difficult to understand a newly encountered accent than a familiar one. Nonetheless, increased exposure to the initially unfamiliar speaking style will usually invoke a subconscious adaptation, a learning effect, making that speech easier to understand. This holds true even for dysarthric speech. Naïve Listeners Expert Listeners Learning Effect from Repeated Exposure to Dysarthric Speech Data - Mean Score Improvement: Round 1 vs. Rounds 2-5 (from ABI Corpus, Birmingham Uni.)

Modelling the Naïve Listener  If the learning effect alters a listener’s perception of a particular individual’s speaking style, is that listener’s judgement still representative of the naïve listener? If the learning effect introduces an inevitable bias, can a computer model be built which behaves like an “eternal” naïve listener (i.e. never adapting to an unfamiliar speaking style and therefore always consistent in assessment)? If the learning effect introduces an inevitable bias, can a computer model be built which behaves like an “eternal” naïve listener (i.e. never adapting to an unfamiliar speaking style and therefore always consistent in assessment)? Possible Solution: Using HMM Models to Emulate the Naïve listener A hidden Markov Model (HMM) is, essentially, a statistical representation of a speech unit at the phone/word/utterance level. HMM models are “trained” by analysing the acoustic features of multiple utterances representing the specified speech unit. A hidden Markov Model (HMM) is, essentially, a statistical representation of a speech unit at the phone/word/utterance level. HMM models are “trained” by analysing the acoustic features of multiple utterances representing the specified speech unit. Multiple Speech Samples from multiple speakers

Goodness of Fit  Once trained, an HMM word model can be used to estimate the likelihood that a given speech sound could have actually been produced by that word model. This likelihood is called a goodness of fit (GOF) and can be expressed as a log likelihood, e.g (or simply -35). Mr. HMM Model, could you’ve been my daddy? Hmm, with a log likelihood of , I’m not so sure… The more acoustically dissimilar an utterance is from what the IE has been trained on, the lower the GOF score

Using Forced-Alignment GOF scoring to measure Intelligibility  Since two of the FDA intelligibility tests require the repetition of words/phrases from a pre-selected vocabulary, HMM utterance models can be built for these words/phrases.  Furthermore, the incoming speech can be matched to the corresponding utterance model to determine the goodness of fit. This matching of a speech sample to a specific utterance model and only that model is called forced alignment.  We hypothesise that force-aligning a speech sample with its corresponding “everyman” word model will yield GOF scores which are systematically related to that speech sample’s intelligibility. When HMMs are used in this way, we call them intelligibility estimators.

…so, how does it work in practice?  IE utterance models are trained on normal speech from a variety of speakers and a range of GOF scores for normal speech test data is established: typically between -5 and -10.  Ranges have been established for moderate and low intelligibility (which, in an FDA diagnostic context = dysarthric) speech, typically with GOF scores between -11 and -20 (moderately intelligible) and < -20 (low intelligibility). These scores are relative to the maximum likelihood utterance (i.e. the speech file with the highest GOF score) in the IE’s training set.

Sample GOF scores GOF scores for isolated single words GOF scores for short sentence utterances

Problem: How do we make IEs truly naïve?  “Everyman’ HMM utterance models are not really ‘everyman’, it’s not feasible to train them on speech data representing all the world’s anglophone accents. In this experiment, the utterance models have been trained on speech principally from the South Yorkshire region, thus accents not represented in the HMM training data could receive GOF scores which do not truly reflect that speech sample’s intelligibility as perceived by a naïve listener.  A non-trivial problem: Certain anglophone accents, due to their prestige, are more universally intelligible than others, e.g. Estuary English and RP, while others are a lot less intelligible internationally (e.g. the Glaswegian accent). What mix of accents should be used to train an HMM word model to make it truly representative of a ‘typical’ naïve listener?

Objective #2: Overall Diagnosis  After collecting data from all the 28 FDA sub-tests, how do we arrive at a dysarthria sub-type diagnosis?  Usually by template matching and symptom categorisation (e.g. “At-rest tasks performed better than in-speech tasks? If so, spastic dysarthria most likely”).  Can these processes be automated? Yes, via a neural network combined with an expert system. The neural network does the basic pattern matching while the rule-based expert system attempts to disambiguate diagnostic information not directly represented in the FDA letter grades. Uncontrollably Rapid Speech Rate? Hypokinetic Dysarthria Most likely of 5 types Slow Speech Rate? Extrapyramidal Dysarthria less likely than other 4 types Yes No Flaccid Dysarthria most likely of 5 types Yes No Example of CFDA Expert system rule- based data disambiguation

Diagnostic Accuracy of Hybrid System

The automated diagnostic system will even tell you why it came to a given decision…

Future Work  Acquisition of HMM Technology which (for the Intelligibility Estimator) doesn’t have prohibitively high license fees.  Collection of dysarthric data to build an FDA- specific dysarthric speech database.  More interviews with experienced speech therapists to increase the diagnostic expert system’s knowledge database.  Results of NHS Field Trials of the CFDA application