Some Further Considerations in Combining Single Case and Group Designs Ann P. Kaiser Vanderbilt University

Today’s Talk Defining Your Program of Research Placing Single Case Designs in Programs of Research Optimizing Your Program of Research

Defining Your Program of Research

Programs of Research: Parts and Wholes What is the focus of your research? What is the goal of your research program? What major components contribute to this goal? What are the research questions within goals? What types of research are needed to address these goals? What designs are needed to address research question?

Focus Components Eric Carter, VU Used with Permission

Types And Research Questions

Placing SCD Studies in Your Program of Research

Contributions of Experimental Designs Single Case Group Demonstration of effects on targeted outcomes Child Adult Individual response to treatment Component analyses Comparisons of discrete variations Dosage Instructional approaches Context Analysis of generalization, maintenance Demonstration of effect of complex, multicomponent interventions Comparisons Treatment/no treatment Treatment comparison Multiple outcome measures Effects on global outcome measures Achievement, standardized tests Standardized quantification of outcomes with confidence estimates Examination of predictors, moderators, mediators in relation to outcomes Individual characteristics Implementation fidelity Examination of longitudinal effects Long term outcomes, controlling for development

Optimizing the contributions of SCD in research program Select questions that can be addressed can be tested in strong SCD Choose a design that meets contemporary standards Functional relation Length of conditions Randomization Blinding Statistical analysis Plan replication, sequential studies, linked questions Include measures that will be used in RCT in addition to strong SCD measures SCR Institute 2018 Kaiser

General logic model EMT en Espanol SCR Institute 2018 Kaiser

SCD Studies

Contributions of SCD Studies to EMT En Espanol research program Independent variable Deliverable in a relatively brief intervention Multi-component vs selected components Dependent variable Construct validity in relation to proposed global outcome Responsive to short term intervention Student/Child outcomes, Adult/implementer outcomes or both Context Ecological validity: setting, implementer Cultural validity Where is new, specific evidence needed? Parent training in Spanish at home Training materials in Spanish Adaptations for parent background (linguistic input) Adaptations for parenting style (interaction style) Child outcomes in English and Spanish -Observed productive language - [Probes for receptive language] Context adaptations Redefining play context Add book reading Inclusive of multiple caregivers Length of intervention Measurement constraints Social validity

Contributions of SCD studies to evidence supporting EMT en Espanol Limitations of Evidence Demonstration of effects on training parents Established fidelity of treatment (Teach Model Coach Review Parent Training) Short term, discrete child outcomes with modest effects Intervention required to insure dosage for child outcomes- not yet specified Limited assessment of maintenance Limited social validity Partial test of the model

Optimizing Your Program of Research

Develop Expertise Design Expertise Content Expertise Measurement Expertise Implementation Expertise Research Questions

Place SCD Studies in an Implementation Science Framework 1. Make a clear statement of beliefs, values and principles guiding the intervention 2. Develop, test and define specific treatment components that promote consistency of implementation 3. Develop and test treatment decision-making protocols 4. Pay attention to how context facilitates treatment 5. Plan for continuous improvement Adapted from Fixen, et al, (2005) Implementation Research: A Synthesis of the Literature

Consider Journal Reporting Standards When Designing Studies APA Journal Reporting Standards (JARS APA 2018) Specific journals standards for SCD What Works Clearing House Standards change – stay ahead of the curve!

Consider Your Timeline and Resources Does your program of research have a timeline? How much will a study cost? Time People: research staff, participants Travel, supplies, materials, incentives SCD studies can be costly How long will a study take? IRB Community/context arrangements Training staff: data collection, implementation Running the study Analyzing data and writing up

Collaborate Build a research team in your setting Technical capacity: data collection, coding, implementation, analysis, writing People (human capacity) Establish context, values, goals, support Seek collaborators with different skills and resources but a shared agenda, values, work style Work with a mentor and/or affiliate with an established team working on your focus Join a research network in your focus area Reach out to key researchers for expertise Situate your research in you identified community and involve stakeholders

Writing Writing is an integral part of the research cycle You may not understand your results or the limitations of the study until you try to write up the study Unpublished studies are …..[invisible, not helpful, frustrating, excluded from meta analysis, common] People come and go, but data in published studies last forever ( Mark Wolery, personal communication)

Programs of Research: Parts and Wholes What is the goal of your research program? What research is needed to address this goals? What are the key questions ? What designs are needed to address research question? What resources, expertise, collaborations are needed? What is your timeline?

Summary Define the ultimate goal of your program of research – start with the end in mind Develop an overarching model for impact Define questions- what do you need to know? Choose methods to fit the questions Prioritize: most important, possible with available resources Build capacity for your program research Develop expertise Collaborate within and outside your lab Stay current with evolving standards Write to know and to have an impact on both research and practice

For more information: Ann.Kaiser@vanderbilt.edu http://kc.vanderbilt.edu/kidtalk/