1. Research Methods for Counselors COUN 597 University of Saint Joseph Class # 4 Copyright © 2015 by R. Halstead. All rights reserved.

2. Class Objectives  Trochin Chapter 7 - Design  Salkind Chapter 6 - Linear Regression

3. External Validity  We learned in Trochim Chapter 2 that external validity is tied to our sampling procedure.  External validity concerns itself with the degree to which we can draw valid conclusions about a population given the findings of a study on a sample from that population.

4. Internal Validity  Internal validity must also be considered when designing your research.  Internal validity addresses the degree to which one can make accurate inferences about the causal relationship between the dependent and independent variable.  What?

5. Internal Validity - Continued  Parsing the definition of Internal Validity  1) making accurate inferences about the casual relationship  Causal or Not Causal?  2) between the dependant variable (DV) and the independent variable (IV)  IV = that which we manipulate (Type of Tx)  DV = that which we have decided to observe before and after Tx

6. The Nature of Causation  You will remember from the days of your youth (week 1 of this class) that there are certain criteria for causality.  1. Variable A ( the cause) must precede Variable B (the effect).  2. Var A must be empirically correlated with Var B (gunpowder explodes Var A - you hear bang Var B).  3. The relationship between Var A and Var B can not be explained by a third Var that causes Var A and Var B (Ice Cream sales and Sun Screen sales).

7. Internal Validity - Continued  A study with good internal validity will have several components  Well defined and tightly controlled intervention  Valid and reliable measure to help in establishing observations  Good controls in place to insure that what is observed can be explained only by the independent variable(s) and not by some alternative cause  Let’s take a look

8. Internal Validity What you did and what you saw Your program and your observations Program Observations What you do What you see Is the relationship causal between... Alternativecause Alternativecause Alternativecause Alternativecause Trochim, 2001 program-outcome relationship

9. Establishing Cause and Effect Temporal precedence Covariation of cause and effect No alternative explanations CauseEffectthen Time if X, then Y if not X, then not Y ProgramOutcome Causes? Alternativecause Alternativecause Alternativecause Alternativecause Trochim, 2001

10. In Typical Outcome Evaluation  Is taken care of because you control the intervention  Is taken care of because you intervene before you measure outcome  Is the central issue of internal validity -- usually taken care of through your design Temporal precedence Covariation of cause and effect No alternative explanations Trochim, 2001

11. Research Design  When constructing your research design you must take into account all of the factors that go into insuring causality.  This is called research design.

12. What Is Research Design? The structure of research Trochim, 2001

13. Elements of a Design  Observations or measures  Treatments or programs  Groups  Assignment to group  Time Trochim, 2001

14. Observations or Measures Symbolized with an "O". Subscripts are used to distinguish different combinations of measures only if this is necessary. Trochim, 2001

15. Treatments or Programs Symbolized with an "X". Subscripts are used to indicate different programs or combinations of programs. Trochim, 2001

16. Groups Each group of interest on its own line. Trochim, 2001

17. Assignment to Groups R = Random assignment N = Nonequivalent groups C = Assignment by cutoff Trochim, 2001

18. Time Left-to-right movement denotes the passage of time. Trochim, 2001

19. Design Notation Example ROXOROOROXOROO Trochim, 2001

20. Design Notation Example ROXOROOROXOROO Time Trochim, 2001

21. Design Notation Example ROXOROOROXOROO O’s indicate different waves of measurement. Trochim, 2001

22. Design Notation Example ROXOROXOROOROOROXOROXOROOROO Vertical alignment of O’s shows that pretest and posttest are measured at same time. Trochim, 2001

23. Design Notation Example ROXOROOROXOROO X is the treatment. Trochim, 2001

24. Design Notation Example ROXOROOROXOROO There are two lines, one for each group. Trochim, 2001

25. Design Notation Example ROXOROOROXOROO R indicates the groups arerandomlyassigned. Trochim, 2001

26. Design Notation Example RO 1 XO 1, 2 RO 1 O 1, 2 Subscriptsindicate subsets of measures. Trochim, 2001

27. Design Notation Example Pretest-posttest (before-after) Treatment versus comparison group Randomized experimental design ROXOROOROXOROO Trochim, 2001

28. Pre-Post Designs  Randomized experiment (R) – Assigned randomly  Nonequivalent group design (N) – No random assignment but taking advantage of convenient groupings (e.g. classrooms)  Regression-Discontinuity design (RD) – Assigned by cut off scores OXOOOOXOOOO Trochim,2001

29. The Main Issues  Selection threats  Are the groups similar before the treatment?  Related to assignment method in each design

30. Types of Designs Random assignment? Control group or multiple measures? Yes No YesNo Randomized or true experiment? Quasi-experimentNonexperiment Trochim, 2001

31. Factorial Designs  Sometimes a counselor might want to look at more than two groups at time.  In such situations the counselor will employ a factorial design.

32. A Simple Example Group 2 average Group 4 average Group 1 average Group 3 average Time in Instruction Setting 1 hour/week4 hours/week In-class Pull-out Usually, averages are in the cells. Trochim, 2001

33. Questions?

34. 9 * 8 * 7 * 6 * Y5 * 4 * 3 * 2 * 1 * 0 1 2 3 4 5 6 7 8 9 X If we know the value of X we can figure out what the value of Y should be. We are just working the process in reverse to make a prediction. Linear Regression Lets assume we have a correlation of r = 1.00 as indicated by our line of regression.

35. Linear Regression - Predicting Var Y from Var X  We must keep in mind that we are predicting future outcomes from that which is already known to hold in the present. Not talking about cause and effect.  December Clients’ Level of Depression Var X  December Client’s Level of Hopefulness Var Y 5 7 14 16 18 20 20 20 24 26 32 33 37 32 38 28 30 26 24 18 19 14 9 6 11 3

36. Linear Regression - Predicting Var Y from Var X 40 * 35 * 30 * * * 25 * Y 20 * * 15 * 10 * * 5 * * 0 5 10 15 20 25 30 35 40 X The Regression Line for Y and X has the least amount of distance from all actual plotted points. It is for that reason it is called the line of best fit. This provides a good rough estimate. Error in Prediction

37. Linear Regression  The line of linear regression provides us with the smallest possible distances from each of our known plotted coordinates for Var X and Var Y.  This line provides us with a way of make a predictive estimate of hopefulness from a known level of depression.

38. Linear Regression  The distance between each individual data point and the regression line is the Error of Prediction.  The error of prediction reflects the strength (or lack of strength) of correlation between our two variables.  We will always have some error in our prediction unless the correlation is perfect (+ 1.00).

39. Simple Formula for Drawing a Line of Regression  Y’is the predicted score of Y based on known value of X.  b is the slope, or direction, of the regression line.  a is the point at which the line crosses the y-axis.  X is the value being used as the predictor. Y’ = bX + a

40. Summary & Making Connections  With one distribution of scores we compute the Mean, median, and mode as measures of central tendency.  Once we have a mean we can calculate deviations from the Mean and the Standard Deviation (the average distance of all scores from the Mean).  The Standard Deviation squared is the Variance.

41. Summary & Making Connections  With two distributions (two dimensions) we compute the Correlation, Line of Regression, Error of the Estimate (a deviation from the line of regression, and the Standard Error of the Estimate (the average of all the Errors of the Estimate).  The Correlation squared is the Coefficient of Determination (the percent of the variance in one variable that is accounted for by the other variable).