1. Associate Professor Genevieve McArthur Director of the Macquarie University Cognition Clinic for Reading Acting Head of the Department of Cognitive Science ARC Centre for Cognition and its Disorders Macquarie University genevieve.mcarthur@mq.edu.au How to make evidence based decisions about literacy interventions

2. Outline The difference between the truth (fact) and what we are told (fiction) Treatments for “dyslexia” “causal” and “reading” treatments good examples of evidence-based thinking a step-by-step guide to the evidence-based treatment of dyslexia practice

3. The difference between truth (fact) and what we are told (fiction)

5. Why am I banging on about flossing? This represents the size of the gap (i.e., Grand Canyon) between what science and the media tell us about the treatment of dyslexia Steps to close the gap …

6. Dorothy Bishop dystalk.com/talks/60-evaluating-alternative-solutions-for- dyslexia Good examples of evidence-based thinking Ben Goldacre http://www.youtube.com/watch?v=O1Q3jZw4FGs http://www.youtube.com/watch?=_UrhJ8P8iZ8&feature=relate d

7. Commercial treatments for dyslexia Reading treatments“Causal” treatments

8. “treatment x” review“treatment x” meta analysis Yes Go to step 2 No Found Use conclusion Step 1: Find a systematic review (a review of all properly controlled trials of a “treatment”) Reading or spelling? Independent? Peer reviewed? Go to step 2 Not found “treatment x” systematic review

9. Yes Go to step 3 Found Reading or spelling? Independent? Peer reviewed? Not found Control group Significantly bigger reading gains in treated group than control group No difference in reading gains in treated and control groups Go to step 3 No No control group Step 2: DIY systematic review “treatment x” randomised controlled trial “treatment x” study “treatment x” controlled trial

10. Step 3: Use indirect evidence a treatment reading is known to improve reading is known to improve trained skill ? ✗ ✓

11. Step 3a: Find out what skill a treatment trains What does “treatment x” train?

12. Step 3b: Find out if training that skill improves reading

13. Step 3: Use indirect evidence a treatment reading is known to improve reading is known to improve trained skill ? ✓ ✓ ✗ ✗

14. Step 4: Strike out No systematic review No studies for DIY systematic review No indirect evidence Conclusion: no scientific evidence Do not try treatment Evidence-based decisionStab in dark Try treatment

15. Beware of the brain Brain data cannot tell us if an educational treatment works or not If a program did not improve reading, but did change the brain, would you buy it? If a program did improve reading, but did not change the brain, would you not buy it? Focus on the behaviour (i.e., reading behaviour) and not the brain

16. Beware of the anecdote Biased Subjective Confounded by other factors (e.g., treatments) Cognitive dissonance

17. Examples “Causal” treatments Reading treatments

18. Example 1

19. Yes Use conclusion Step 1: Find a systematic review Reading or spelling? Independent? Peer reviewed? Found fast forword systematic review fast forword meta analysisfast forword review little or no effect

20. Example 2

21. Yes Use conclusion Step 1: Find a systematic review Reading or spelling? Independent? Peer reviewed? Found Irlen lenses systematic review Irlen lenses meta analysisIrlen lenses review little or no effect

22. Example 3

23. arrowsmith reviewarrowsmith meta analysis Go to step 2 Step 1: Find a systematic review Not found arrowsmith systematic review

24. Step 2: DIY systematic review arrowsmith randomised controlled trial arrowsmith study arrowsmith controlled trial Go to step 3 No studies

25. Step 3: Use indirect evidence reading is known to improve trained skill ✗ ✓ reading is known to improve ?

26. Step 3a: Find out what skill a treatment trains What does arrowsmith train? The Arrowsmith Program identifies, intervenes and strengthens the weak cognitive capacities that affect learning. Students are able to capitalize on their increased learning capacities and after a three or four year program can function without special education assistance or program accommodations. Upon completion of the program some students may require one to two years to gain experience using their newly strengthened cognitive capacities and some students may need tutoring initially to bring academic skills to grade level given the limited amount of time within the program to address academic skill deficits.

27. Step 3b: Find out if training that skill improves reading

28. Step 3: Use indirect evidence reading is known to improve trained skill ? cognitive capacity ? reading is known to improve ?

29. Step 4: Strike out No systematic review No studies for DIY systematic review No indirect evidence Do not try treatment Conclusion: no scientific evidence Evidence-based decisionStab in dark Try treatment ✓ ✓ ✓ ✓ ✓ ✓

30. If you want a backup or its all a bit too much...

31. Outline Treatments for dyslexia “causal” and “reading” treatments good examples of evidence-based thinking a step-by-step guide to the evidence-based treatment of dyslexia practice

32. Good luck

33. Example 4

34. reading eggs reviewreading eggs meta analysis Go to step 2 Step 1: Find a systematic review Not found reading eggs systematic review

35. Step 2: DIY systematic review reading eggs randomised controlled trial reading eggs study reading eggs controlled trial Go to step 3 No studies

36. Step 3: Use indirect evidence reading is known to improve trained skill ✗ ✓ a treatment reading is known to improve ?

37. Step 3a: Find out what skill a treatment trains What does reading eggs train? phonics

38. Step 3b: Find out if training that skill improves reading

39. phonics reviewphonics meta analysis Yes Use conclusion Step 3b: Find a systematic review of that skill Reading or spelling? Independent? Peer reviewed? Found phonics systematic review moderate to large effects in dyslexia

40. Step 3: Use indirect evidence a treatment reading is known to improve reading is known to improve trained skill ✓ ✓ phonic s