1. Assessment & Technology UH-M COLLEGE OF EDUCATION COE Outreach & Technology

2. Electronic Exhibit Room Systematic assessment of evidence of student learning at multiple points in program Program assessment data compiled to an internal website NCATE reviewers access at their leisure Data remain available between reviews Easy for program faculty to maintain

3. Program Assessment Assessed by: – Candidates (exit surveys, course evaluations) – Alumni (survey, focus groups) – Employers – Mentor Teachers – COPR Process – Learned Societies – Candidate Learning Outcomes Review Assessing Programs by Assessing Candidates

4. Candidate Assessment – Evidence Collection = Portfolio Grade Reports Exam Scores Work Samples Faculty Observation Summaries Students Work Samples – Candidate Portfolio Tools PowerPoint (hyperlinks, external files, branching) Task Stream – online CD-R

5. Assessing Learning Outcomes 1. Define Program Objectives 2. Define Points of Measurement 3. Define Evidence for Objectives 4. Define Rubric for Assessing Evidence 5. Delineate Who Evaluates and When

6. Example (part 1: Program defines Objectives) ABC Program Objective 1: (Knowledge) Candidates know... (Skill) Candidates are be able to … (Disposition) Candidates exhibit… Objective 2: Objective 3: Objective 4:

7. Secondary Program Objectives SECONDARY PROGRAM OBJECTIVES EDUC 401EDPSYC 311/611 EDUC 4 02 EDUC 404 TECS 440 SPED 445 ETEC 414 EDUC 405-6 Introduction to Secondary Education Educ Psychology FieldMethodsMulti- Cultural Special Education Educ Tech- nology Student Teaching Professional Legal Responsibilities xxx X x X Foundations of Secondary Education X xxxx

8. Example (part 2: Program Chooses Points of Measurement) Program will Assess Candidates – Beginning (defined: immediately upon admission) – Middle (defined: conclusion of EDUC XXXX course and/or prior to student teaching) – End (defined: conclusion of field experience XXXX)

9. Example (part 3: Program assigns evidence to objectives) Objective 1: Professional Legal Responsibilities The teacher candidate demonstrates an understanding of (knowledge) and ability to apply (skill) and model (disposition) legal responsibilities expected of professional educators.

10. Sample artifacts Program ObjectivesSuggested Artifact EvidenceRubric Scale 1. Professional Legal Responsibilities The teacher candidate demonstrates an understanding of and ability to apply and model legal responsibilities expected of professional educators. Case study responses Reflective journals and logs Performance evaluation Evidence of ability to maintain required reports, records, and legal documents IEP from a case study report 1 2 3

11. Example (part 4: Program defines rubric scale for evidence) 3Target: Evidence reflects in-depth knowledge and understanding of standard; outstanding data and evidence of application 2Acceptable: Evidence indicates knowledge and understanding of standard; satisfactory data and evidence of application 1Unacceptable: Evidence shows little or inadequate knowledge of standard; limited data and evidence of application

12. Example (part 5: Program states who will measure and when) Candidate Outcomes Review Faculty assigned to review candidate outcomes Review Committee determines program completion for candidate Candidate outcomes aggregated Summary data provided to Associate Dean on cohort

13. Example (part 6: Composite candidate scores defined, measured) Summarize each Candidates Assessment Mid-Point Assessment: – e.g. Overall Unacceptable: 1 or more unacceptables – e.g. Overall Acceptable: 0 unacceptables, <4 superiors – e.g. Overall Superior: 0 unacceptables, 5 or more superior scores

14. Example (part 7: Summary data submitted) ABC Program Midpoint Assessment 2004 Incomplete Overall Unacceptable Overall Acceptable Overall Superior 112312

15. NCATE Review Website (mock-up)

17. Measurements: 5 year period

18. Use of Technology Candidate use to collect and present evidence Program use to assess candidate learning Program use of aggregated data for program review College use to maintain data overtime for accreditation purposes

19. Challenges Requires a shift in thinking: from grades to authentic assessment of learning outcomes Program objectives must be made explicit Faculty agreement on rubrics and scales Need to identify ways to manage the process Technology must be helpful, not burdensome