1. Some Further Considerations in Combining Single Case and Group Designs
Ann P. Kaiser
Vanderbilt University

2. Today’s Talk Defining Your Program of Research
Placing Single Case Designs in Programs of Research
Optimizing Your Program of Research

3. Defining Your Program of Research

4. Programs of Research: Parts and Wholes
What is the focus of your research?
What is the goal of your research program?
What major components contribute to this goal?
What are the research questions within goals?
What types of research are needed to address these goals?
What designs are needed to address research question?

5. Focus
Components
Eric Carter, VU
Used with Permission

6. Types
And
Research
Questions

7. Placing SCD Studies in Your Program of Research

8. Contributions of Experimental Designs
Single Case
Group
Demonstration of effects on targeted outcomes
Child
Adult
Individual response to treatment
Component analyses
Comparisons of discrete variations
Dosage
Instructional approaches
Context
Analysis of generalization, maintenance
Demonstration of effect of complex, multicomponent interventions
Comparisons
Treatment/no treatment
Treatment comparison
Multiple outcome measures
Effects on global outcome measures
Achievement, standardized tests
Standardized quantification of outcomes with confidence estimates
Examination of predictors, moderators, mediators in relation to outcomes
Individual characteristics
Implementation fidelity
Examination of longitudinal effects
Long term outcomes, controlling for development

9. Optimizing the contributions of SCD in research program
Select questions that can be addressed can be tested in strong SCD
Choose a design that meets contemporary standards
Functional relation
Length of conditions
Randomization
Blinding
Statistical analysis
Plan replication, sequential studies, linked questions
Include measures that will be used in RCT in addition to strong SCD measures
SCR Institute 2018 Kaiser

10. General logic model EMT en Espanol
SCR Institute 2018 Kaiser

11. SCD Studies

12. Contributions of SCD Studies to EMT En Espanol research program
Independent variable
Deliverable in a relatively brief intervention
Multi-component vs selected components
Dependent variable
Construct validity in relation to proposed global outcome
Responsive to short term intervention
Student/Child outcomes, Adult/implementer outcomes or both
Context
Ecological validity: setting, implementer
Cultural validity
Where is new, specific evidence needed?
Parent training in Spanish at home
Training materials in Spanish
Adaptations for parent background (linguistic input)
Adaptations for parenting style (interaction style)
Child outcomes in English and Spanish
-Observed productive language
- [Probes for receptive language]
Context adaptations
Redefining play context
Add book reading
Inclusive of multiple caregivers
Length of intervention
Measurement constraints
Social validity

13. Contributions of SCD studies to evidence supporting EMT en Espanol
Limitations of Evidence
Demonstration of effects on training parents
Established fidelity of treatment (Teach Model Coach Review Parent Training)
Short term, discrete child outcomes with modest effects
Intervention required to insure dosage for child outcomes- not yet specified
Limited assessment of maintenance
Limited social validity
Partial test of the model

14. Optimizing Your Program of Research

15. Develop Expertise Design Expertise Content Expertise
Measurement Expertise
Implementation Expertise
Research Questions

16. Place SCD Studies in an Implementation Science Framework
1. Make a clear statement of beliefs, values and principles guiding the intervention
2. Develop, test and define specific treatment components that promote consistency of implementation
3. Develop and test treatment decision-making protocols
4. Pay attention to how context facilitates treatment
5. Plan for continuous improvement
Adapted from Fixen, et al, (2005) Implementation Research: A Synthesis of the Literature

17. Consider Journal Reporting Standards When Designing Studies
APA Journal Reporting Standards (JARS APA 2018)
Specific journals standards for SCD
What Works Clearing House
Standards change – stay ahead of the curve!

18. Consider Your Timeline and Resources
Does your program of research have a timeline?
How much will a study cost?
Time
People: research staff, participants
Travel, supplies, materials, incentives
SCD studies can be costly
How long will a study take?
IRB
Community/context arrangements
Training staff: data collection, implementation
Running the study
Analyzing data and writing up

19. Collaborate Build a research team in your setting
Technical capacity: data collection, coding, implementation, analysis, writing
People (human capacity)
Establish context, values, goals, support
Seek collaborators with different skills and resources but a shared agenda, values, work style
Work with a mentor and/or affiliate with an established team working on your focus
Join a research network in your focus area
Reach out to key researchers for expertise
Situate your research in you identified community and involve stakeholders

20. Writing Writing is an integral part of the research cycle
You may not understand your results or the limitations of the study until you try to write up the study
Unpublished studies are …..[invisible, not helpful, frustrating, excluded from meta analysis, common]
People come and go, but data in published studies last forever ( Mark Wolery, personal communication)

21. Programs of Research: Parts and Wholes
What is the goal of your research program?
What research is needed to address this goals?
What are the key questions ?
What designs are needed to address research question?
What resources, expertise, collaborations are needed?
What is your timeline?

22. Summary
Define the ultimate goal of your program of research – start with the end in mind
Develop an overarching model for impact
Define questions- what do you need to know?
Choose methods to fit the questions
Prioritize: most important, possible with available resources
Build capacity for your program research
Develop expertise
Collaborate within and outside your lab
Stay current with evolving standards
Write to know and to have an impact on both research and practice

23. For more information: Ann.Kaiser@vanderbilt.edu