1. SPHS 5780, LECTURE 6: PSYCHOMETRICS, “STANDARDIZED ASSESSMENT”, NORM-REFERENCED TESTING

2. pSYCHOMETRICS

3. psychometricS “psycho” +“metric”
Who is responsible for assessing psychometric adequacy?
Is this the case even when a test or procedure is mandated?
Who is affected when a psychometrically inadequate measure is used?

4. psychometricS
Discussion: What makes a test or diagnostic procedure “good”?

5. psychometricS
When assessing psychometric adequacy, we are assessing rigor of assessment
Validity: Testing actually assesses what you think it is assessing (what it is supposed to be assessing)
Reliability: Testing is consistent across time and across different clients

6. psychometricS
Is assessment of psychometric adequacy (reliability and validity) needed for:
Formal, norm-referenced tests?
Criterion-referenced tests?
Behavioral observation?

7. Psychometrics
Rigor of assessment for informal testing
Validity: Clear definition of the test domain, on which experts agree, as evidenced through “mastery by masters” and “non-mastery by non-masters”
Reliability: Consistency in
decision outcome
“distance” from cut-off score
score on closely related versions of measure
Rigor or assessment for formal tests will be addressed in more detail in the current lecture

8. standardization

9. Standardization insures that all people taking the test…..
receive the same experience
are expected to perform the same task with the same materials
receive the same amount of assistance from the examiner
are evaluated according to a standard set of criteria
WELL CONCIEVED STANDARDIZED TESTS ALLOW FOR MEANINGFUL COMPARISON AMONG CHILDREN
Ideally, standardized test are developed by giving the test to a large group of children so that they (the test makers) may compute an acceptable range of variation in the scores for the age range covered.
If the same materials, procedure, scoring are not used on all children than the results will have limited comparability across children.
Example: put together a three pieces puzzle
some would show child first
some might not show the child first
some would impose a 1 minute strategy others would have limitless times.
Strict adherence to standardized procedure is fundamentally important
Problem doing this with some handicapped children---Informal use of standardized test.
…regardless of time or place of administration, regardless of clinician

10. Standardization
“Standardized test” is a term commonly used to refer to norm-referenced tests. Why? Does this mean that other types of tests are not standardized?

11. Standardization
How does standardization support
a test’s validity?
a test’s reliability?

12. Review of the formal /informal distinction

13. Purpose of formal vs. Informal Testing
To determine eligibility for services, a norm-referenced test needs to be sensitive to the presence of a disorder (not miss those who have disorder) and specific in determination of referrals (not refer those who do not have a disorder). The same requirements do not hold for informal testing. Do you see why?

14. Examples of formal vs. Informal Testing
Formal: CELF Informal: Augmentative communication evaluation

15. NORM-REFERENCED MEASURES CRITERION-REFERENCED MEASURES
Function
Identify clients who need help
Plan the nature of that help
Normative group performance
Establish group of scores against which client is compared
Establish performance standard (performance “criterion”)
NOTE: Performance standard may be client’s own past performance!
Interpreta-
tions of client’s performance
Relative to range of performance of others
Relative to standard (“criterion”): mastery vs. non-mastery
When to use
When norms are available
When broad content (“trait”) is of interest
When norms are unavailable or inappropriate
When specific skills/behaviors are of interest
Specific Dx
Focus Tx planning

16. NORM-REFERENCED MEASURES CRITERION-REFERENCED MEASURES
Level of detail for given area
Low detail (broad area)
High detail (narrow area)
Summary scores
Converted scores
Raw scores
Choice of test items
Those items on which test-takers perform variably
Those items always passed by masters
Those items always failed by non-masters
Decision to be defended
Relative ranking compared to normative group
Dichotomous (mastery, non-mastery)

17. Formal (Norm-referenced) Tests: Nature

18. Purpose and procedure of norm-referenced tests
Fundamental purpose is to rank individuals
“to determine if an individual obtains a score similar to the group average or, if not, how far away from average the score is” (H&P, M&P)
“(to determine) if there is a problem, or a significant enough difference from standard performance to warrant concern with regard to normalcy”
Performance summarized with reference to the “standardization sample” (normative sample), through conversion of raw scores
percentile scores
standard scores

19. Advantages of Norm-Referenced Tests
Objectivity Replicability Elimination of unwanted / uncontrolled variation

20. Disadvantages of Norm-Referenced Tests
For some domains (e.g. language) it is difficult to find a language test that is:
valid
reliable
The normative sample needs to represent your client
NRTs do not accommodate cultural variation
NRTs do not accommodate individual variation

21. Norm referenced tests cannot:
Measure treatment progress
Guide creation of therapy goals, through analyzing individual test items
Richly describe performance
Accommodate cultural variation
Discussion of WHY this is the case….

22. Lecture 06a ends here this is where material for exam 2 ends