1. The Practice of Science and the Science of Practice in Autism Spectrum Disorders Samuel L. Odom FPG Child Development Institute University of North Carolina

2. Goals of the Presentation Describe the origins of evidence-based practice Describe the origins of evidence-based practice Examine process and criteria for identifying evidence-based practice in behavioral intervention research Examine process and criteria for identifying evidence-based practice in behavioral intervention research Discuss issues related to implementation of EBPs Discuss issues related to implementation of EBPs Describe strategies for supporting implementation Describe strategies for supporting implementation

3. Animal Models for Intervention

4. Neuroscience

5. Autism and ASD: Agreeing on Terminology and Characteristics Definition and Diagnosis (Autistic Disorder) Definition and Diagnosis (Autistic Disorder) DSM-IV Criteria often the standard for diagnosis criteria DSM-IV Criteria often the standard for diagnosis criteria Core features Core features Social competence Social competence Communication Communication Repetitive behavior/need for sameness Repetitive behavior/need for sameness Autism Spectrum Disorder Autism Spectrum Disorder Autistic Disorder Autistic Disorder Pervasive Developmental Disorder Not Otherwise Specified Pervasive Developmental Disorder Not Otherwise Specified Asperger’s Disorder Asperger’s Disorder

6. Practice of Science in Autism Intervention Research Goals have been: Goals have been: Cure autism Cure autism Remediate or ameliorate condition for children and families Remediate or ameliorate condition for children and families Social science intervention research has focused on two classes of interventions in its research literature Social science intervention research has focused on two classes of interventions in its research literature Comprehensive treatment models Comprehensive treatment models Focused intervention models Focused intervention models

7. What Do We Mean By Practice? Comprehensive Models vs. Focused Interventions Comprehensive models are multi-component programs designed to positively and systematically affect the lives of children with ASD and their families Comprehensive models are multi-component programs designed to positively and systematically affect the lives of children with ASD and their families Focused interventions are procedures that promote individual skills or learning within a specific skill area. Focused interventions are procedures that promote individual skills or learning within a specific skill area.

8. Origins of Evidence Based Practices Current emphasis may be traced to Cochrane’s (1972) concern about the lack of use of research in health care practice Current emphasis may be traced to Cochrane’s (1972) concern about the lack of use of research in health care practice Formation of Cochrane Collaborative to conduct systematic reviews of effects of health care interventions Formation of Cochrane Collaborative to conduct systematic reviews of effects of health care interventions Evidence-based medicine coined as a term at McMaster University in 1980’s Evidence-based medicine coined as a term at McMaster University in 1980’s Sackett and colleagues were primary early advocates for evidence-based medicine Sackett and colleagues were primary early advocates for evidence-based medicine

9. Neither old hat nor impossible to practice Neither old hat nor impossible to practice Not at “cook-book” medicine Not at “cook-book” medicine Not a cost cutting medicine Not a cost cutting medicine Not restricted to randomized trials and meta-analyses Not restricted to randomized trials and meta-analyses Sackett (1996) qualifications of EBM

10. Movement of “evidence-based” into Education Campbell Collaboration formed (in the US) in 1990 to conduct systematic reviews of educational and social policy practices Campbell Collaboration formed (in the US) in 1990 to conduct systematic reviews of educational and social policy practices What Works Clearinghouse funded by Institute of Education Sciences operates through CC What Works Clearinghouse funded by Institute of Education Sciences operates through CC Evidence for Policy and Practice Information Center (EPPIC) at the University of London Institute of Education was created in 1993 Evidence for Policy and Practice Information Center (EPPIC) at the University of London Institute of Education was created in 1993 Center for Evidence-Based Practice at Orelena Puckett Institute in North Carolina Center for Evidence-Based Practice at Orelena Puckett Institute in North Carolina

11. Efforts to Identify Evidence-Based Practices/Professional Associations Child-Clinical Section of Division 12 of the American Psychological Association Child-Clinical Section of Division 12 of the American Psychological Association CEC-Division for Research CEC-Division for Research National Association for School Psychology (empirically supported interventions) National Association for School Psychology (empirically supported interventions) ASHA ASHA DEC Recommended Practices DEC Recommended Practices

12. APA Task Force on Promotion and Dissemination of Psychological Procedures: Well Established At least two well-conducted group-design studies, conducted by different investigators At least two well-conducted group-design studies, conducted by different investigators Superior to pill placebo or alternative treatment Superior to pill placebo or alternative treatment Equivalent to well established treatment Equivalent to well established treatment Large series of single-case design studies (n>9) that use good experimental design and compare the intervention to another treatment Large series of single-case design studies (n>9) that use good experimental design and compare the intervention to another treatment Treatment manuals used for the intervention Treatment manuals used for the intervention Clearly specify sample characteristics Clearly specify sample characteristics

13. Attempts to Examine Evidence- Based Practices for Children with Autism Spectrum Disorder States States New York Department of Health New York Department of Health California Department of Human Resources California Department of Human Resources Research Organizations Research Organizations National Academy of Sciences National Academy of Sciences National Autism Center—National Standards Project National Autism Center—National Standards Project Professional Organizations Professional Organizations National Academy of Pediatrics National Academy of Pediatrics Scholars Scholars Rogers Rogers Dawson & Osterling Dawson & Osterling Odom, Brown, et al. (2003) Odom, Brown, et al. (2003)

14. What Counts As Evidence?

15. Peer-reviewed, refereed journal articles Peer-reviewed, refereed journal articles Report research Report research Clearly identified children with ASD and/or families were participants Clearly identified children with ASD and/or families were participants Methodologies Methodologies Experimental Group Designs Experimental Group Designs Quasi experimental designs Quasi experimental designs Single subject designs Single subject designs

16. Quality Indicators for Experimental and Quasi-Experimental Research Gersten, Compton, Fuchs, Greenwood, Innocenti, & Coyne (2005)

17. What do we mean by Experimental Group Research? Randomized Clinical Trials (Randomized Experimental Group Designs) Randomized Clinical Trials (Randomized Experimental Group Designs) Quasi-Experimental Designs Quasi-Experimental Designs

18. Quality Indicators for Single Subject Design Horner, Carr, Halle, McGee, Odom, & Wolery (2005)

19. Major Quality Indicator Categories Experimental Control Experimental Control Description of participants and setting Description of participants and setting Dependent variable Dependent variable Independent variable Independent variable Baseline Baseline External Validity External Validity Social Validity Social Validity

21. Criteria for Evidence-Based Practices for Focused Interventions (NPDC-ASD) Two high quality randomized experimental group design or quasi-experimental group designs that rule out selectivity and other threats to internal validity Two high quality randomized experimental group design or quasi-experimental group designs that rule out selectivity and other threats to internal validity Five high quality single subject design Five high quality single subject design At least three different researchers in different locations At least three different researchers in different locations Each study has at least three demonstrations of experimental control Each study has at least three demonstrations of experimental control

22. Criteria for Evidence-Based Practices for Focused Interventions (NPDC-ASD) Combined evidence Combined evidence One RCT or high quality quasi- experimental design One RCT or high quality quasi- experimental design At least three high quality single subject designs At least three high quality single subject designs

23. Review of Literature Began by looking at outcomes that related to the core features of autism Began by looking at outcomes that related to the core features of autism Social Social Communication Communication Repetitive and problem behavior Repetitive and problem behavior Adaptive behavior Adaptive behavior Academic skills Academic skills Identified and grouped interventions that addressed these skills Identified and grouped interventions that addressed these skills Looked for similar interventions across skill domains and age levels Looked for similar interventions across skill domains and age levels

25. Focused-Interventions Identified Behavioral intervention practices Behavioral intervention practices Prompting Prompting Time delay Time delay Reinforcement Reinforcement Task Analysis and Chaining Task Analysis and Chaining

26. Behavioral Interventions to Decrease Interfering Behaviors Positive behavior support Positive behavior support Functional Behavior Assessment Functional Behavior Assessment Differential reinforcement of alternative behavior Differential reinforcement of alternative behavior Extinction Extinction Response interruption/redirection Response interruption/redirection Stimulus Control Stimulus Control Functional Communication Training Functional Communication Training

27. Focused Interventions Discrete trial training Discrete trial training Naturalistic intervention Naturalistic intervention Pivotal response training Pivotal response training Self-management Self-management

28. Focused Interventions Visual supports Visual supports Individualized work systems Individualized work systems Video modeling Video modeling Computer-assisted instruction Computer-assisted instruction VOCA VOCA

29. Focused Interventions Social skills training Social skills training Peer mediated intervention Peer mediated intervention Social Stories Social Stories Picture exchange communication system (PECS) Picture exchange communication system (PECS)

31. Evaluation of Comprehensive Treatment Models for Individuals with ASD Odom, Boyd, Hall, & Hume (2009)

32. Evaluation Purpose Provide information upon which to make decisions Provide information upon which to make decisions School districts to make decisions about adoption School districts to make decisions about adoption Families choose a model for their children Families choose a model for their children Systematic review of “model features” Systematic review of “model features” Critical evaluation Critical evaluation

33. Comprehensive Treatment Strategies Multiple components (e.g., child-focused instruction, family-focused support) Multiple components (e.g., child-focused instruction, family-focused support) Broad scope (i.e., they address development domains representing the core features of ASD) Broad scope (i.e., they address development domains representing the core features of ASD) Intensity (i.e., they often occur over an entire instructional day or in multiple settings such as a school/clinic and home) Intensity (i.e., they often occur over an entire instructional day or in multiple settings such as a school/clinic and home) Longevity (i.e., they may occur over months or even years). Longevity (i.e., they may occur over months or even years). Replication in the US Replication in the US

34. Evaluation of all CTMs Located all articles in publicly available locations (e.g., journals, book chapters, computer searches, public presentations, etc.) Located all articles in publicly available locations (e.g., journals, book chapters, computer searches, public presentations, etc.) Searched websites for information Searched websites for information Interviewed every CTM developer Interviewed every CTM developer CTM developers send additional materials for our review CTM developers send additional materials for our review Some not publicly available Some not publicly available

37. Evaluation of six features of the programs Operationalization Operationalization Fidelity of Implementation Fidelity of Implementation Replication Replication Outcomes reported Outcomes reported Quality of the research methodology Quality of the research methodology Supplemental research on focused intervention features of the CTM Supplemental research on focused intervention features of the CTM

41. Review by Two Evaluators All trained on two CTMs, up to an 80% criteria All trained on two CTMs, up to an 80% criteria Primary evaluator constructed the portfolio and completed evaluation Primary evaluator constructed the portfolio and completed evaluation Secondary evaluator completed second review of each portfolio Secondary evaluator completed second review of each portfolio When disagreements occurred, resolved through discussion When disagreements occurred, resolved through discussion Inter-rater agreement was 83% (exact agreement) Inter-rater agreement was 83% (exact agreement)

43. The Green CTMs Denver Model Denver Model LEAP LEAP Lovaas Institute Lovaas Institute May Institute May Institute Princeton Child Development Institute Princeton Child Development Institute

44. Emerging Evidence Autism Partnerships Autism Partnerships CARD CARD Children’s Toddler Program Children’s Toddler Program Douglass Developmental Center Douglass Developmental Center Pivotal Response Treatment (PRT) Pivotal Response Treatment (PRT) Responsive Teaching Responsive Teaching SCERTS SCERTS STAR STAR TEACCH TEACCH

45. Conclusions Strongest feature of CTM literature is operationalization Strongest feature of CTM literature is operationalization Fidelity documented but not in rigorous manner Fidelity documented but not in rigorous manner Efficacy and effectiveness studies are limited but growing Efficacy and effectiveness studies are limited but growing Difficulty for classroom-based models Difficulty for classroom-based models Generational issues Generational issues Models are evolving Models are evolving