1. Correlation & Prediction REVIEW Correlation BivariateDirect/IndirectCause/Effect Strength of relationships (is + stronger than negative?) Coefficient of determination (r 2 ); Predicts what? Linear vs Curvilinear relationships

2. Inferential Statistics Used to infer sample characteristics to a population

3. Table 5-2 Variable Classification IndependentDependent Presumed causePresumed effect The antecedentThe consequence Manipulated/measured by researcher Outcome (measured) Predicted fromPredicted to PredictorCriterion XY

4. Common Statistical Tests Chi-Square Determine association between two nominally scaled variables. Independent t-test Determine differences in one continuous DV between ONLY two groups. Dependent t-test Compare 2 related (paired) groups on one continuous DV. One-Way ANOVA Examine group differences between 1 continuous DV & 1 nominal IV. Can handle more than two groups of data.

5. What Analysis? IVDVStatistical Test 1 Nominal Chi-Square 1 Nominal (2 groups) 1 continuoust-test 1 Nominal (>2 groups) 1 continuousOne-Way ANOVA

6. Some Examples Chi-Square Gender and knee injuries in collegiate basketball players (Q angle) Independent t-test Differences in girls and boys (independent groups; mutually exclusive) on PACER laps Dependent t-test Pre and Post measurement of same group or matched pairs (siblings) on number of push-ups completed One-Way ANOVA Major (AT, ES, PETE; IV >2 levels ) and pre-test grade in this class

7. Norm-Referenced Measurement HPHE 3150 Dr. Ayers

8. Topics for Discussion Reliability (variance & PPM correlation support reliability & validity) Consistency Repeatability Validity Truthfulness Objectivity Inter-rater reliability Relevance Degree to which a test pertains to its objectives

9. Reliability Observed, Error, and True Scores Observed Score = True Score + Error Score ALL scores have true and error portions True scores are impossible to measure

10. Reliability THIS IS HUGE!!!! Reliability is that proportion of observed score variance that is true score variance TIP: use algebra to move S 2 t to stand alone as shown in formula above (subtract S 2 e from both sides of equation ) S 2 o = S 2 t + S 2 e

11. Desirable reliability >.80 There is variation in observed, true & error scores Error can be +(↑ observed scores) or –(↓ observed scores) Error scores contribute little to observed variation Error score mean is 0 S 2 o = S 2 t + S 2 e Validity depends on reliability and relevance Observed variance is necessary Generally, longer tests are more reliable (fosters variance)

12. Table 6-1 Systolic Blood Pressure Recordings for 10 Subjects Subject Observed BP = True BP + Error BP 1103105-2 2117115+2 3116120-4 4123125-2 5127125+2 61251250 7135125+10 8126130-4 9133135-2 101451450 Sum (  )125012500 Mean (M)125.0125.00 Variance (S 2 )133.6116.716.9╣ S11.610.84.1 ╣ Se is square root of S 2 e

13. Reliability Coefficients Interclass Reliability Correlates 2 trials Intraclass Reliability Correlates >2 trials

14. Interclass Reliability (Pearson Product Moment) Test Retest (administer test 2x & correlate scores) See Excel document (Norm-ref msmt examples)Excel document Time, fatigue, practice effect Equivalence (create 2 “equivalent” test forms) Odd/Even test items on a single test Addresses most of the test/retest issues Reduces test size 50% (not desirable); longer tests are > reliable Split Halves Spearman-Brown prophecy formula

15. Index of Reliability The theoretical correlation between observed scores and true scores High I of R = low error Square root of the reliability coefficient If r=.81, I of R=.9 Compared to the Coefficient of Determination: r 2 (shared variance)

16. I of R vs C of Det. If r=.81 I of R =? C of Det=?

17. Reliability So What? Find a friend and talk about: 1 thing you “got” today 1 thing you “missed” today; can they help?

18. Reliability REVIEW Inferential Infer sample findings to entire population Chi Square (2 nominal variables) t-test (1 nominal variable for 2 groups, 1 continuous) ANOVA (1 nominal variable for 2 + groups, 1 continuous)

19. Correlation Are two variables related? What happens to Y when X changes? Linear relationship between two variables Quantifies the RELIABILITY & VALIDITY of a test or measurement

20. Reliability (0-1;.80 + goal) All scores: observed = true + error r xx =S 2 t /S 2 o proportion of observed score variance that is true score variance Interclass reliability coefficients (correlates 2 trials) Test/retest time, fatigue, practice effect Equivalent reduces test length by 50% Split-halves Index of Reliability Tells you what? Related to C of D how?

21. Standard Error of Measurement RELIABILITY MEASURE S=standard deviation of the test r xx’ =reliability of the test Reflects the degree to which a person's observed score fluctuates as a result of measurement errors

22. EXAMPLE: Test standard deviation=100r=.84 SEM = =100( .16) =100(.4) =40

23. SEM is the standard deviation of the measurement errors around an observed score EXAMPLE: Average test score=500SEM=40 68% of all scores should fall between 460-540 (500+40) 95% of all scores range between: ?420-580

24. Standard Error of Estimate (reflects accuracy of estimating a score on the criterion measure) VALIDITY MEASURE Standard Error Standard Error of Prediction

25. Standard Errors both are standard deviations SE of Measurement (reliability) SE of Estimate (criterion-related validity)

26. Factors Affecting Test Reliability 1)Fatigue ↓ 2)Practice ↑ 3)Subject variability homogeneous ↓, heterogeneous ↑ 4)Time between testing more time= ↓ 5)Circumstances surrounding the testing periods change= ↓ 6)Test difficulty too hard/easy= ↓ 7) Precision of measurement precise= ↑ 8)Environmental conditions change= ↓ SO WHAT? A test must first be reliable to be valid

27. Validity Types THIS SLIDE IS HUGE!!!! Content-Related Validity (a.k.a., face validity) Should represent knowledge to be learned Criterion for content validity rests w/ interpreter Use “experts” to establish Criterion-Related Validity Test has a statistical relationship w/ trait measured Alternative measures validated w/ criterion measure Concurrent: criterion/alternate measured same time Predictive: criterion measured in future Construct-Related Validity Validates theoretical measures that are unobservable

28. Methods of Obtaining a Criterion Measure Actual participation (game play) Skills tests, expert judges Perform the criterion (treadmill test) Distance runs, sub-maximal swim, run, cycle Heart disease (developed later in life) Present diet, behaviors, BP, family history Success in grad school GRE scores, UG GPA

29. Interpreting the “r” you obtain THIS IS HUGE!!!!

30. Correlation Matrix for Development of a Golf Skill Test (From Green et al., 1987) Playing golf Long puttChip shotPitch shotMiddle distance shot Drive Playing golf 1.00 Long putt.591.00 Chip shot.58.471.00 Pitch shot.54.37.351.00 Middle distance shot.66.55.61.401.00 Drive -.65-.62-.48-.52-.791.00 What are these? Concurrent Validity coefficients

31. Interpret these correlations Actual golf score Putting Trial 1 Putting Trial 2 Driving Trial 1 Driving Trial 2 Observer 1 Observer 2 Actual golf score 1.00 Putting T1.781.00 Putting T2.74.831.00 Driving T1.58.21.251.00 Driving T2.68.25.30.701.00 Observer 1.48.34.40.43.381.00 Observer 2.39.30.41.47.35.501.00 What are these? Concurrent Validity coefficients Criterion

32. Interpret these correlations Actual golf score Putting Trial 1 Putting Trial 2 Driving Trial 1 Driving Trial 2 Observer 1 Observer 2 Actual golf score 1.00 Putting T1.781.00 Putting T2.74.831.00 Driving T1.58.21.251.00 Driving T2.68.25.30.701.00 Observer 1.48.34.40.43.381.00 Observer 2.39.30.41.47.35.501.00 What are these? Reliability coefficients

33. Interpret these correlations Actual golf score Putting Trial 1 Putting Trial 2 Driving Trial 1 Driving Trial 2 Observer 1 Observer 2 Actual golf score 1.00 Putting T1.781.00 Putting T2.74.831.00 Driving T1.58.21.251.00 Driving T2.68.25.30.701.00 Observer 1.48.34.40.43.381.00 Observer 2.39.30.41.47.35.501.00 What is this? Objectivity coefficient

34. Concurrent Validity This square represents variance in performance in a skill (e.g., golf)

35. Concurrent Validity The different colors and patterns represent different parts of a skills test battery to measure the criterion (e.g., golf)

36. Concurrent Validity The orange color represents ERROR or unexplained variance in the criterion (e.g., golf) Remember: ↑error = ↓ validity Error

37. Concurrent Validity ACDB Consider the Concurrent validity of the above 4 possible skills test batteries

38. Concurrent Validity ACDB Which test battery would you be LEAST likely to use? Why? D – it has the MOST error and requires 4 tests to be administered

39. Concurrent Validity ACDB Which test battery would you be MOST likely to use? Why? C – it has the LEAST error but it requires 3 tests to be administered

40. Concurrent Validity ACDB Which test battery would you use if you are limited in time? A or B – requires 1 or 2 tests to be administered but you lose some validity