2. Computer-supported intervention for children with language and literacy problems Ludo Verhoeven In collaboration with Eliane Segers and Rosemarie Irausquin University of Nijmegen

3. Possibilities computer Connecting speech, orthography, semantics Virtual reality Hypertekst Communication Adaptivity Responsivity Motivational value

4. Digital learning environments Computer-assisted instruction –Drills & practice –Games Open learning environments –(Automatic) feedback –Communication

5. Computer-supported intervention Computer-assisted instruction –Early language intervention (kindergarten) –Intervention for poor readers Open learning environment –Language in the content areas

6. Early intervention software Schatkist met de muis Storybook reading –Story telling –Story illustrations –Semantic meaning –Word blending Language play –Multimedial songs –Book making –Postcard writing –Rhyme and word segmenting –Word making

7. Overview of a CDROM (bold arrows show adaptivity)

8. Postcard writing

9. Rhyme and word synthesis: b-el Attention to word sounds Isolating beginning sound Word blending

10. Grapheme booklet: computer keeps track of individual progress

11. Word maker

12. Effects Positive effects –Vocabulary –Text comprehension –Phonological awareness Effects equally strong for L1 and L2 learners Effects for children with SLI –Rhyme –Phonemic awareness –Role of speech manipulation???

13. Research Tallal, Merzenich & associates Children with SLI: –Problems with fast formant transitions in synthetic speech (1974, 1981) –Lengthening signal > better discrimination (1973) –Lengthening formant transition > better discrimination (1975, 1980) –New algorithm: lengthening whole signal + amplifying fast formant transitions up to 20dB –Commercial program: Fast Forword (1996: Effectful after 100 hours in four weeks of training)

14. Difficulties with fast formant transitions yearauthorssubject groupspeech materialconclusion 1974Tallal & Piercy12 aphasic children 12 controls age 6.9-9.3 years old /ba/ - /da/; formant transition 43 msec aphasics have more problems than controls 1980aTallal, Stark, Kallmann & Mellits 35 developmental dysphasics 38 controls average age: 6.8 years old /ba/ - /da/ as in Tallal and Piercy 1974. Synthetic copies of natural utterances /ba/ - /be/ /bi/ - /dae/, /de/ - /di/; formant transition varying naturally dysphasics have more problems than controls 1981Tallal & Starksame as above/ba/ - /da/, /da/ - /ta/, /e/ - /ae/, /dab/ - /daeb/, /sa/ - /sta/, /sa/ - /fa/ dysphasics have more problems than controls in /ba/-/da/, /da/-/ta/ and also /sa/-/fa/. 1989Reed23 reading disabled 23 'normals' average age: 8.9 /ba/ - /da/, /e/ - /ae/,reading disabled have problems in /ba/ - /da/ as opposed to controls. 1992Leonard, McGregor & Allen 8 SLI, 8 normals. 4.5-5.6 years old /ba/ - /da/, /dab/ - /daeb/, /i/ - /u/, /dab-i-ba/ - /dab-u- ba/, /das/ - /daf/ SLI's have more problems than controls not in /dab/-/daeb/ and /i/-/u/

15. Benefits of speech manipulation 1975Tallal & Piercy12 aphasic children 12 controls (6.8-9.3) /ba/ - /da/; formant transition extended from 40 to 80msec positive effects 1980Frumkin & Rapin20 dysphasic children 9 control children average age 9.6 /ba/ - /da/ and /a/ - /u/ Synthetic speech, 2nd and 3rd formant 40msec or 80 msec; total length 250 msec positive effects for subgroup 1980bTallal, Stark, Kallman & Mellits 14 developmental dysphasics 23 normally developing age 5-9 /ba/ - /da/ Synthetic copies of natural utterances /be/ - /be/, /bi/ - /dae/, /d  / - /di/; formant transition varying naturally results aphasics dependent on syllable speed and ISI time 1982Alexander & Frost24 children with language/speech problems (7.2-11.6) /ba/ - /da/. Formant transitions: 80, 70, 60 and 40 msec. positive effects 1984Blumstein, Tartter, Nigro & Statlender 16 aphasics, 6 controls2 synthetic /ba/ - /da/ - /ga/ continua with formant transition 65 and 85 msec. no effects 1985Riedel & Studdert- Kennedy 12 adult aphasics. average age 55 /ba/ - /da/ Formant transition 30 and 82 msec. no effects 1985Tallal, Stark & Mellits 26 developmental dysphasics same as Tallal and Stark 1981unclear 1996Stark & Heinz11 children with output disorders 21 children with SLI 22 controls (6-10 yrs) /ba/ /da/ Klatt synthesis. Formant transition 30 to 80 msec. in steps of 10. positive effect for children with expressive and receptive problems 1999Bradlow et al.32 children with learning problems 72 controls (6 –16 yrs) two /da/ - /ga/ continua 40 msec and 80 msec formant transition no effects

16. Training studies using speech manipulation 1996Tallal et al.22 SLI average age 7.4 Computer games with or without speech manipulation (Fast Forword) positive effects 1999Habib, et al.12 dyslexics age 10-12 Listening exercises with or without speech manipulation positive effects 2001Gillam et al.4 SLI average age 7 ;3 Fast Forword vs Laureate Learning software no differential effects 2003Troia & Whitney 37 LD: 25 FFW/12 C average age 9;4 Fast Forwordoverall effect on expressive language only

17. Study 1: Speech manipulation Participants –21 children with SLI vs 24 NLA controls (5 yrs) Stimulus set –Five contrasts: b-p, d-t, v-w, h-g, b-d –60 word pairs, e.g., buik-duik (12x60 items) Speech manipulation: –Normal speech –Amplifying fast formant transitions –Slowing down speech signal –Amplification + slowing down

18. Oscillogram normal vs amplified speech (enhanced fast transitional elements): /buik/ - /duik/

19. Study 1 (continued) Results –Normal children perform better than SLI children –+/- voice is more complex than place contrasts –No effect of speech manipulation Conclusion –No replication of effects reported by Tallal & Piercy

20. Study 2: Training with natural speech manipulation Participants (kindergarten) –36 SLI children: 24 experimental vs 12 control Training: rhyme and word blending Normal speech (N1=12) Manipulated speech (N2=12) Procedure –Pretest-posttest-retention test

21. Results study 2

22. Conclusions Study 2 Program is effective for children with SLI Significant retention effects No effect for speech manipulation

23. Study 3: Training with synthetic speech manipulation Participants (kindergarten) –19 SLI children and 24 NLA children Stimulus set: minimal word pairs –Normal –Slowing down entire speech signal (like in FFW) –Slowing down just formant transitions

24. Results study 3

25. Conclusions Study 3 Program is effective for NLA and SLI Positive effect speech manipulation for SLI No difference between slowing down entire signal or formant transitions

26. General conclusions SLI children have difficulties in phonological tasks Phonological training is helpful for SLI children Role of speech manipulation is inconclusive Limitations: small N, limited hours of training

27. Dyslexia Phonological deficit –Phonological memory –Phonemic segmentation –Access to phonology Sensory defects –Processing brief sensory cues –Processing rapidly changing sequences –Limited use of temporal information

28. LEESLADDER: Adaptive computer program for poor readers cd-rom 1: alphabetic principle, phonological awareness, decoding of CVC words cd-rom 2: automatisation of reading and spelling of simple word structures + basic reading comprehension cd-rom 3: reading/spelling monosyllabic words with consonant clusters and specific orthographic patterns (- eeuw, -ooi, -ng, etc.) + reading comprehension cd-rom 4: decoding multisyllabic words + advanced reading comprehension

29. Pupil database Pupil ID Learning history Teacher part Parameters Planning abilities Play types (Sub)abilities Sources database graphemes words Sentences, etc.. Pupil part Adaptive navigation module

30. Basic 1 Basic 2 Reading and spelling 1 Reading and spelling 2 Reading and spelling 3 Grapheme discovery Sound discovery Grapheme- and phoneme knowledge Type & copy Word closure Auditory synthesis Letter flashing Grapheme order correct Simple spellingWord reading correct Grapheme order fast Auditory word dictation Picture-word Word-picture Word rows 1 Word reading fast Visual word dictation Word rows 2Flash words

31. Basic 1 Basic 2 Reading and spelling 1 Reading and spelling 2 Reading and spelling 3 Grapheme discovery Phoneme discovery Grapheme and phoneme knowledge Type and copy Word closure Auditory synthesis Flash letters Grapheme order correct Elementary spelling Word reading correct Grapheme order fast Auditory word dictation Picture-word Word-picture Word rows 1 Worden reading fast Visual word dictation Word rows 2Flash words Letter test

32. Effects on phonological awareness

33. Effects on word decoding

34. Correlations between training intensity and reading gains Exercises per weekDays per week CVC word reading test.53 *.56 * Word reading test.54 *.58 * Pseudoword reading test.62 *.59 *

35. Speed vs comprehension training

36. General discussion Age: plasticity of the brain Contents of training Synthetic vs natural speech N of hours of training (minimum 100?) Effects: phonology vs information processing Necessity of control groups

37. Open Learning Environment: facilitating language and knowledge construction

38. Communication

39. Text sample

40. Communication: reaction types Reaction types: (1) (dis)approval, (2) question about the text, (3) personal, (4) reasoning, (5) suggestion (6) hello/goodbye, (7) rest category

41. Online spelling feedback Conditions: Handwriting (H), Typing (T), Spellingchecker (C), Spelling suggestions (S)

42. Perspective Children need to learn how to communicate in open learning environment Spelling development can be supported by means of online feedback Children with learning problems equally benefit from the environment CIA can be integrated in open learning environments