CHASING CHAllenging Speech training In Neurological patients by interactive Gaming Utrecht, November 12, 2015

Dysarthric patients: diminished intelligibility Speech therapy  Short period of frequent sessions with speech therapist  After that, therapy stops Therapy can have positive effects  Increased loudness (and not pitch)  But intelligibility, articulation lags behind, and  Therapy should be intensive & continuous Dysarthric speech  Training speech at home using the computer: extended intensified therapy, reduce costs  Game principles: motivate to keep practicing (more intensive)

The CHASING team CLST Project coordination Speech technology Effect studies Speech therapy Game design and development

1.How to design and develop an optimal game for speech training (natural user interaction)? Goal: increased intelligibility 2.Which are the advantages and disadvantages of various feedback moves? E.g. explicit vs. implicit (game) feedback 3.What are the effects of the game both in terms of efficacy and user satisfaction (compared to EST)? Intelligibility assessment : objective & subjective User satisfaction Motivation CHASING: Research questions

Activities so far Literature studies on:  Dysarthric speech and its correlates  Use of ASR technology for evaluating dysarthric speech  Feedback techniques in speech training  Use of games in therapy Studies on dysarthric speech:  Find acoustic-phonetic features that characterize hypokinetic and/or spastic dysarthria  Compare different subjective evaluation methods for dysarthric speech ASR technology:  Research on applying a new engine (Deep Learning / DNNs)  Preparing speech database for development and testing

Activities so far Setting up advisory group: two patients and one game expert Game development:  Focus sessions and interviews with patients and therapists  Brainstorming sessions with project group  Presentation game concepts to advisory group  Elaboration of game concept selected  Testing game with project group  Testing game by advisory group: many important improvements  Adaptation game  Testing improved version with larger group of users  Fine tuning game

Only title ASR & PED for eLearning + eTherapy 7 CHASING

ASR & PED for eLearning + eTherapy 8

CHASING ASR & PED for eLearning + eTherapy 9

Only title ASR & PED for eLearning + eTherapy 10 Add Speech Technology for key phrases

 Conducting pilot experiments for study 1  Recruiting subjects for study 1  Recruiting ‘actors’ (e.g. student assistants) for playing the game with patients  Conducting study 1  Integrating ASR into game for key phrases feedback on pronunciation, articulation, intelligibility Plans for immediate future

More information  Project page Radboud University Nijmegen:  Project page Waag Society:  Project page Waag Society: voor-neurologische-patienten-via-interactieve-games- chasing/ voor-neurologische-patienten-via-interactieve-games- chasing/

More information  Project page Radboud University Nijmegen:  Project page Waag Society:  Project page Waag Society: voor-neurologische-patienten-via-interactieve-games- chasing/ voor-neurologische-patienten-via-interactieve-games- chasing/

The CHASING team CLST

The CHASING team CLST Helmer Strik – project leader CHASING Mario Ganzeboom – PhD student Marjoke Bakker – postdoc researcher Toni Rietveld – phonetician, methodologist Catia Cucchiarini – phonetician -Project coordination -Speech technology -Effect studies

The CHASING team Lilian Beijer – postdoc researcher -Expertise speech therapy -Effect studies

The CHASING team Sabine Wildevuur – head Creative Care Lab Jurre Ongering [was: Paulien Melis] – project leader Waag Douwe-Sjoerd Boschman – concept developer Lodewijk Loos - programmer -Concept development -Game design and development

Work plan Duration: January 2014 to December 2017

Issues  Which game principles are suitable for dysarthric patients with Parkinson’s disease or stroke?  What are the physical and cognitive capabilities of these patients in working with a game?  How can speech exercises be integrated in a game in a motivating way?  How can speech intelligibility of dysarthric speakers be assessed automatically?  Types of feedback: implicit vs. explicit immediate vs. delayed visual, auditory, …