1. Assessment Developing an Assessment

2. Assessment Planning Process Analyze the environment Agency, clients, TR program, staff & resources Define parameters Function, content, implementation strategy Select or develop assessment Establish assessment protocols Train staff & interns on protocols Peterson & Stumbo, 2000

3. Assessment Implementation Process Review assessment protocol Prepare for assessment Administer assessment to client Analyze or score assessment results Interpret results for placement into programs Document results of assessment Reassess client as necessary/monitor progress Peterson & Stumbo, 2000

4. Selection of Assessment Procedure Determine the purpose Specify the exact content to be covered Identify other selection criteria Search assessment resources Compare possible procedures to selection criteria Select assessment

5. Don’t Create an Agency Assessment When: A commercial assessment is readily available that fits purpose & clients The staff has limited expertise or interest in test development or reliability & validity Programs are not based on a comprehensive program design Stumbo & Peterson, 2009

6. Reality “In actual practice, more therapeutic recreation specialists rely on agency-specific assessments than on nationally available tests.” Peterson & Stumbo, 2000, p. 221 Why? Content of commercial assessments rarely matches content of programs or agencies

7. Rationale for Teaching Development to Undergraduate Students “Many individuals feel that assessment development and validation is a complex enough process that it should not be advocated as a task for the typical TRS. Individuals with bachelor’s degrees may not have the conceptual and statistical expertise for development and validation. Another view states that TRSs are more likely to develop their own assessments (good or bad) than buy commercial assessments so instruction in correct methodology is important.” (Stumbo, 2002, p. 163)

8. Developing an Assessment Select the assessment content Comes from content of program Based on current literature, needs of clients, and TR models or theories Develop list of programs designed for a specific group Define client goals

9. Developing an Assessment Determine the assessment purpose Placement into programs Basic screening, identifying problems, narrow problems, reassess problems Understand client group Disabilities, education, backgrounds, etc.

10. Developing an Assessment Determine type of assessment Interviews, observations, self- administered surveys, record reviews How many clients assessed? How much staff time is available for assessment? What are the skills of staff? What will yield the best information to place in programs?

11. Developing an Assessment Select the client outcomes to be assessed Outline the assessment content Make a two-way chart Make sure assessment is representative of content of program See p. 170

12. Developing an Assessment Select the type of items to use Closed-ended or open-ended Yes/no, true/false, matching, multiple choice, rating, & ranking = closed Fill-in-the-blank, short answer, & extended answer = open Match the items to specific client outcomes

13. Developing an Assessment Make the final selection of items See Stumbo, 2002, p. 181-182 Norm group Validity & reliability

14. Developing an Assessment Make manual/protocol Directions for administration, scoring, interpretation, and reporting Reliability & validity See Stumbo, 2002, p. 182-183 Pilot test & make revisions

15. What’s Needed in a Protocol Preparation Environmental conditions Resources Length of time Administration Instructions for interview, observation, test What to do in exceptional circumstances

16. What’s Needed in a Protocol Scoring How answers to each question s/b scored How the total assessment is to be scored (e.g. subscores, total) Analysis and Interpretation How items or test is to be interpreted for program placement

17. What’s Needed in a Protocol Reporting/documentation How results are to be recorded How results are to be reported Stumbo & Peterson, 2009

18. Do Create an Agency- Specific Assessment When: An appropriate commercial assessment can’t be located TR programs or clients have unique characteristics A comprehensive program is designed and results in a clear view of program content to be used in an assessment

19. Do Create an Agency- Specific Assessment When: TR staff have time to make an assessment TR staff know how to design an assessment TR staff know how to design an assessment protocol Stumbo & Peterson, 2009