1. Assessment Procedures for Counselors and Helping Professionals, 7e © 2010 Pearson Education, Inc. All rights reserved. Chapter 7 Selecting, Administering, Scoring, and Interpreting Assessment Results

2. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-2 Selecting Assessment Instruments & Strategies 1. Identify the type of information needed 2. Identify available information 3. Determine the methods for obtaining information 4. Search assessment resources 5. Evaluate and select an assessment instrument or strategy

3. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-3 Identify the Type of Information Needed Any assessment instrument or strategy is useful only if it provides the information required for the assessment. Counselors determine what information to collect based on the purpose for assessment.

4. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-4 Identify Available Information Pre-existing assessment information about clients can include: intake questionnaires biographical data preliminary diagnoses school grades previous assessment results

5. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-5 Determine the Methods for Obtaining Information Counselors can choose from both formal and informal assessment instruments and strategies

6. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-6 Search Assessment Resources References sources Mental Measurement Yearbook Tests in Print Test Critiques Tests

7. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-7 Publisher catalogs provide important test information including: Cost of Materials and Scoring Types of Scoring Services While publisher catalogs are a valuable source of information, bias may exist. Catalogs should be used along with other sources of information. Test Publisher Catalogs

8. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-8 Test manuals are a primary source of information when selecting tests The Standards for Educational and Psychological Testing (1999) provides a list of key components that should be included in test manuals. Test Manuals

9. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-9 Academic journals dedicated to the exploration of testing and assessment can be valuable resources in test selection. Individual journal articles may explore the use of specific tests in various settings. Research Literature

10. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-10 Professional Organizations American Educational Research Association Association for Assessment in Counseling & Education National Center for Research on Evaluation, Standards, and Student Testing (CRESST) National Council on Measurement in Education (NCME) American Counseling Association American Psychological Association

11. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-11 Online test locator services can be valuable resources for selecting tests. Buros Institute (http://www.unl.edu/buros/)http://www.unl.edu/buros/ Educational Testing Service (www.ets.org)www.ets.org Internet Resources

12. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-12 Directories of Unpublished Instruments Directory of Unpublished Experimental Measures (2007) Measures for Psychological Assessment: A Guide to 3,000 Original Sources and Their Applications (1974)

13. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-13 Evaluate and Select an Assessment Instrument or Strategy 1. What is the purpose of the instrument? Who is the intended population? 2. What sample of the population was used in instrument development, validation, and norming? 3. Are the results of the instrument reliable? 4. Do the instrument’s results have evidence of validity? 5. Does the instrument’s manual provide clear and detailed instructions about administration procedures? 6. Does the manual provide sufficient information about scoring, interpreting and reporting results? 7. Is the instrument biased?

14. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-14 Evaluate and Select an Assessment Instrument or Strategy cont. 8. What level of competency is needed to use the instrument? Test User Qualifications A-level: no advanced training required B-level: master’s degree in counseling, psychology, education or related area, or have completed specialized training or expertise C-level: doctorate in psychology, education, or a related field, or licensure.

15. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-15 Evaluate and Select an Assessment Instrument or Strategy cont. 9. What practical issues should be considered for this instrument? a. Time required for administering the instrument. b. Cost of the instrument c. Format d. Readability e. Ease of Administration f. Ease of Scoring g. Ease of Interpretation

16. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-16 Modes of Administering Instruments: Self-Administered Individually Administered Group Administered Computer Administered Video Administered Audio Administered American Sign Language Nonverbal Administering Assessment Instruments

17. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-17 Before Administration Counselor should be familiar with the instrument. Review manual, forms, other materials. Be familiar with the content of the instrument. Review directions for administering the instrument. For standardized tests: schedule date, room, facilities; obtain booklets, answer sheets, pencils, other materials Obtain informed consent from examinees Provide orientation for examinees

18. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-18 During Administration Final check of materials, lighting, ventilation, seating arrangements, work space, pencils, etc. For standardized instruments, follow administration procedures and deliver instructions verbatim. Establish rapport with client. Be alert to problems and client behaviors during administration.

19. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-19 After Administration Collect materials Count booklets and answer sheets Pack materials according to stated procedures Record any incidents.

20. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-20 Scoring Assessment Instruments Hand-scoring Computer-scoring Sent to publisher for scoring Self-scored by client

21. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-21 Scoring Performance Assessment Performance assessment: portfolios, projects, tasks, open-ended exercises. No clear-cut right or wrong answers. Scored using a scoring rubric.

22. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-22 Scoring Errors Errors in Hand scoring: Assigning incorrect values to individual responses. Incorrectly recoding responses Incorrectly converting raw scores to standard scores Calculation errors Errors in Computer scoring: Inputting data incorrectly Incomplete erasures or stray marks (when using optical scanners)

23. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-23 Interpreting Assessment Results Determine if score is norm-referenced or criterion-referenced. Explain the meaning of various standard scores and their implications to the test taker. Understand factors that can affect assessment interpretation: 1. Psychometric factors 2. Test taker factors 3. Contextual factors

24. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-24 Interpreting Assessment Results cont. Computer-generated reports or narratives: Canned interpretations based on test results. May be very complex Not stand-alone interpretations: computers cannot incorporate uniqueness of the client Counselors ultimately accountable for accuracy of interpretations, even when using computer- generated reports.

25. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-25 Consider any major differences between the norms or comparison groups and the actual test takers and any differences in test administration practices or familiarity with the specific questions on the tests. Avoid using a single test score as the sole determinant of decisions about test takers. Avoid using tests for purposes not specifically recommended by the test developer unless evidence is obtained to support an alternate use. Report test results accurately and clearly and in a manner than is understood by the test taker. Routinely review a sample of the test answer sheets to verify the accuracy of the initial scoring. Employ systematic procedures to verify the accuracy and consistency of computer scoring of answer sheets. Standards for Scoring and Interpreting Tests

26. Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved. 1-26 Obtain a separate and independent verification that appropriate scoring rules and normative conversions are used for each person tested. Verify as accurate the computation and conversion of raw scores and the conversion of raw scores to normative or descriptive scales prior to release of such information to the test taker or to users of the test results. Check routinely for accuracy of manual or computer reports of test results. The person performing this task must be qualified to recognize inappropriate or impossible scores. Develop and use systematic and objective procedures for observing and recording the conditions and behaviors of persons being tested and make this part of the scores or test results that are reported. Label clearly the scores that are reported and the date that a particular test was administered.