1. Study Design Research Methods Professional Development Institute Kali Trzesniewski December 4, 2015

5. Before you start planning Need to know where you are going If you were going to build a hotel BEFORE deciding the look and function of the hotel or making a blueprint buy materials Make an action plan

6. So, don’t start working on Measures Sample Design (e.g., cross-sectional, experimental) Tasks (e.g., activities) Etc Until you have a clear goal

7. What type of evidence will allow me to reach my goal in a convincing way? Study Design Function

8. The function of a study design is to ensure that the evidence obtained enables us to answer the initial question as unambiguously as possible

9. So, when designing a study, ask: Given my research question, what type of evidence is needed to answer the question in a convincing way?

10. Keep in Mind Part of answering your research question in a convincing way… is addressing alternative, rival explanations

11. Types of Research Designs

12. Descriptive oDoDocuments frequencies or levels Correlation oToTests relations between two variables oFoFocused on individual differences oMoMay be cross- sectional or longitudinal Experiment oCoControlled context to test causal impact oFoFocused on group differences oRoRequires random assignment to control for individual differences Quasi- Experiment oNoNaturally occurring experiment oWoWeaker causal impact oFoFocused on group differences oRoRequires control group, but not random assignment Time in EFNEP Amount of Vegetables Consumed Re-enroll in 4-H Participation in County Fair Better Goal Setting Randomly assigned to 4-H Thrive!

13. Research Design Strengths and Limitations Descriptive o Quick and easy o Interpretation not always clear Correlation o Quick and easy o Cannot inform causation

14. Research Design Strengths and Limitations Experiment o Can inform causation o Harder to do o No one wants to be the control o Might not reflect real life Quasi-experiment o Easier than experiment o Cannot make strong causation conclusions o With statistical adjustments (e.g., propensity matching) causation can be suggested

15. The design you use will determine the conclusions you can make

16. Research Question: Is my program effective? Hypothesis: Youth who participate in my program will make better food choices

17. Do youth increase on self-reported food choices pre to post? Do youth who have participated in my program make better food choices than youth who have not participated in the program? Does my program change the food choices youth make? Operationalizing your hypothesis

18. Research design: correlation Conclusion: program resulted in better food choices Why people might not trust the conclusion: change might reflect naturally-occurring growth o Need a control sample to show change isn’t happening in a similar population of non- participants Do youth increase on self-reported food choices pre to post?

19. Research design: quasi-experiment & descriptive Conclusion: program resulted in better food choices Why people might not trust the conclusion: youth who ended up in program might have always had better food choices o Need random assignment or statistical controls Do youth who have participated in my program make better food choices than youth who have not participated in the program?

20. Research design: experiment Conclusion: program resulted in better food choices Why people might not trust the conclusion: people generally trust experiments o Questions usually revolve around generalizability and scaling up Does my program change the food choices youth make?

21. Choosing Which Design to Use

22. What do I want to learn? What will I want to report? Participants, leaders, community members, funders, etc. What might I want to know for program refinement? What will my take-home message be? What figures or tables will I want? Jump to the End

23. What can I feasibly do?

24. Common Mistakes in Research Insufficiently motivated research questions o Is this something people other than me care about? o Will it generate new knowledge? o Does it address a real problem? o Does it impact a sufficient portion of the population?

25. Common Mistakes in Research Unresearchable problems o Can this question be answered through ethically and valid research practices? o Is this feasible for me to do? Favored research methods o Don’t design a study around a research method Blind data mining o Don’t collect data then figure out what to do with it

26. Go forth and research!