1. Appraisal and Its Application to Counseling COUN 550 Saint Joseph College For Class # 3 Copyright © 2005 by R. Halstead. All rights reserved.

2. Class Objectives Review of Descriptive Statistics Distributions Measures of Central Tendency Measures of Variability Correlation Coefficients Reliability

3. Review - Distributions Whenever one attempts to measure some element or quality, differences are observed. When differences are measured numerically (e.g. test scores) and placed side by side in numerical order it is said that they are distributed or a as such form a distribution of score. (e.g. 45 53 56 67 72 75 77 78 83 87 88 94 96) All distributions have qualities that one can describe using descriptive statistics.

4. Review - Qualities of a Distribution Frequency of occurrence for any one score. Frequency distributions have two dimensions. Breadth, reflecting the difference between the highest and lowest score. Height, the number of scores at any one point of measurement. These two dimensions can be visually represented with a histogram, a frequency polygon, or a smoothed curve graphic. The shape of a frequency distribution provides an indication of overall group performance.

5. Review - Measures of Central Tendency Central tendency reflects where most of the cases reside if randomly distributed. Right Click to open link and Watch!! http://www.youtube.com/watch?NR=1&v=5_HVBhwhwV8 Measures of Central tendency include: Mean - The arithmetic average of all scores of a distribution. Median - That point above which one would fine 50% of the cases of a distribution and below which one would fine 50% of the cases of a distribution. Mode - The most frequently occurring score in a distribution

6. Review - Measures of Variability Variability is the degree to which the scores within a distribution disperse from the mean. Range – numerical distance between the highest and lowest scores Variance – The average squared deviation of scores from the mean (or the standard deviation squared) Standard Deviation – The squared root of the variance. It describes the spread or dispersion of scores within a distribution of scores.

7. Correlation Correlation coefficient is a statistical measurement that indicates the strength of relationship between two sets of data (two groups of test scores). In essence, a correlation coefficient tells us how two sets of data “co-relate.” Another way to think about this is how two variables change relative to one another.

8. Correlation - Continued Samples r xy is the correlation between variable x and variable y r height-weight is the correlation between height and weight r SAT-GPA is the correlation between SAT and GPA

9. Correlation - Continued 9 * 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 * 0 1 2 3 4 5 6 7 8 9

10. Correlation - Continued Strength and Direction Perfect Correlation + 1.00 Perfect Correlation - 1.00 None 0.00 Direction Direction - Positive (+) and Negative (-)

11. Absolute Value and Strength Which is the Stronger of the two correlations below? +.50-.70 or Answer: -.70 because the absolute value |.7 | is greater than the absolute value |.5 |.

12. Understanding What Correlation Means - A Rough Guideline General Rule of Thumb Size of the Correlation General Interpretation.8 to 1.0 ---------------------- Very Strong.6 to.8 ---------------------- Strong.4 to.6 ---------------------- Moderate.2 to.4 ---------------------- Weak.0 to.2 ---------------------- Very Weak or No Relationship Good method for a quick assessment

13. Reliability Reliability - Defined as the degree to which test scores are consistent, dependable, and repeatable. As you have read in your text, in order to understand reliability, we must address the issue of error. The reason for this is because reliability is a function of the degree to which test scores are free from error of measurement.

14. The Conceptualization of Error “Hard Science” and “Soft Science” “Hard Science” Research study that attends to forms of measure that tend to be more concrete in nature “Soft Science” Research study that attends to forms of measure that address constructs that lack concrete qualities or must be inferred through secondary observation. In counseling, we are engaged in “soft science” Personality, intelligence, anxiety, depression, etc.

15. Test Score Theory Basic Assumption of Test Score Theory True Score - reflects an accurate measurement of some construct of interest. Every individual taking a test would end up with a True Score except for the fact that no test is perfect. We must always address the fact that with every test there will be some measure of error. The difference between the True Score and the Obtained Score is the Measurement of Error. X (Observed Score) = T (True Score) + E (Error)

16. Test Score Theory and Error Working the Formula X = T + E Observed Score True Score Error X - T = E

17. Conceptualizing Random Error Since every test (or measure) written will have some error, dealing with it in an organized fashion is necessary. Test theory helps in this regard in that that the counselor operates under the assumption that any error encountered will be randomly distributed. When error is randomly distributed, the whole distribution of errors will take the shape of a normal curve.

18. Randomly Distributed Error It is important to keep in mind that when talking of error, what is really expressed is the degree to which one is missing an accurate measurement of some construct (e.g. intelligence, depression, etc.) A distribution of score, therefore, will consist of two components. First, there will be a True Score. Second, there will be lots of errors that have randomly deviated from the True Score.

19. Standard Error of Measurement The Standard Error of Measurement is based on the standard deviation for scores that have deviated from the true score for any given test. By finding the Standard Error of Measurement we can determine with a degree of confidence the true score of a test will fall within a particular range. + 1 SEM = 68 times out of 100 + 2 SEM = 95 times out of 100 + 3 SEM = 99 times out of 100

20. Error and Reliability Error and Reliability are concepts that have an important connection. Reliability – is defined as the degree to which test scores are consistent, dependable, and repeatable. If a test is perfectly reliable we would expect that each time and individual takes that test (assuming stability of the construct of interest e.g. personality trait, anxiety, depression, etc.) that individual score would be exactly the same. Good reliability means that there is very little change in the score each time the test is administered. In other words, the observed score deviates very little from the true score. There is little error in the measure.

21. Sources of Error There are many sources of Error Systematic Error - reflects a false picture because of some flaw in the system. Acquiescence Social desirability Culture bias Random Error - Inconsistencies inherent to any form of measurement

22. Sources of Error Avoiding Error Select instruments that are valid and reliable Look at elements that might suggest bias Select appropriate instruments Conduct a small pilot study

23. Establishing Test Reliability Methods for Establishing Reliability Test-Retest Reliability Parallel Forms or Alternate Forms Reliability Split-Half Reliability Each of these methods rely on the use of correlation. The author of the test would conduct one or more testing methods for establishing reliability and then calculate a correlation coefficient between the two forms of the test. In this case one would be looking for a high positive correlation between the two forms of the measure.

24. Internal Consistency All the measures of internal consistency evaluate the extent to which the different items on a test measure the same ability or trait of interest. Establishing internal consistency is also established through correlation coefficients referred to as Coefficient Alpha. Two common examples are as follow: Kuder-Richardson Cronbach Alpha