Office of Research, Evaluation, and Assessment April 19, 2008

The CTL Comprehensive Web-based Assessment System - Multiple Measurements Across Five Transitions - Multiple Measurements Across Five Transitions - Fairness, Consistency, and Accuracy of Measures - Standards Alignment - Formative and Summative Evaluation - Data Management, Accessibility, and Utility - Analyses Confirm Candidate Performance and Success - The Professional Culture of Accountability

Data are regularly and systematically collected, compiled, summarized, analyzed, and reported publicly to multiple audiences for the purpose of improving candidate performance, program quality, and unit operations. THE ASSESSMENT SYSTEM THE ASSESSMENT SYSTEM DATA MANAGEMENT, ACCESSIBILITY, AND UTILITY Summaries are also shared, discussed, and reported in numerous ways at each level of the unit's governance organization, and are made public through the CTL/OREA web sites. Feedback is sought at every level: candidate, faculty, PEAB, administration, as well as the public. Consistent with the conceptual framework's constructivist philosophy, the CTL assessment system has dynamically evolved over the past decade as a result of relevant and meaningful experiences. The latest edition of the system is comprehensively designed to be purposely redundant in the measurement of standards, flexible enough to meet specific program requirements, and robust enough to provide unit-wide analyses for the purpose of improving program and unit (CTL) operations. Multiple assessment measures are collected using a transitional timeline. Data are collected, aggregated, and reported at both the program level and unit level for both initial and advanced programs using CTL, state, and national standards. Data reports and summaries are used by individual faculty to assess their own instruction; by candidates to assess their own learning and development; by program faculty and coordinators to assess program effectiveness; and by the OREA to assess unit operations.

MULTIPLE MEASUREMENTS ACROSS FIVE TRANSITIONS: A comprehensive assessment system systematically produces data on applicant qualifications, candidate proficiencies, and the performance of graduates… Assessment System Measures by Transition Assessment System Measures by Transition Aligned with Standards

There is a clear explicit connection between the assessment of candidates and professional, state, and institutional standards for the role in which the educator is being prepared. Standards are Aligned in Measurements Standards are Aligned in Courses, Artifacts, Rubrics, and Reports Standards are Aligned in Summary Data Standards are Aligned in Aggregated Data STANDARDS ALIGNMENT State, Professional, and Institutional Standards are Aligned Standards are Aligned in Program Reports Standards are Aligned in Unit Report and Assessment Evaluations

ANALYSES CONFIRM CANDIDATE PERFORMANCE AND SUCCESS Teacher Residency Principal and Program Administrator Professional and Pedagogical Knowledge and Skills -vs- GradesProfessional and Pedagogical Knowledge and Skills -vs- Grades (Pro. Sequence) Summary of Candidate Success(Pro. Sequence) Summary of Candidate Success Pre-Autumn Self Evaluation Mid Quarter Assessment of Knowledge and Skills (Student Teaching) Data show the clear relationship of assessments to candidate outcomes. Documentation of candidate performance verifies learner expectations have been met by all candidates. Professional Growth Plan

FORMATIVE AND SUMMATIVE EVALUATION Formative Measures - GPA (Full and Conditional Admissions) - Student Teaching GPA - Disposition Inventory - Knowledge and Skills on LiveText - Informal and Formal Student Teaching Observations Summative Measures - WEST B - Faculty Referrals - Final Student Teaching Evaluation - Performance-based Pedagogy Assessment - First and Third Year Teacher Surveys - EBI Principal and Teacher Surveys - Placement Surveys - WEST E - Certification Checklist - Admissions Checklist Teacher Residency Data are regularly and systematically collected, compiled, summarized, analyzed and publicly reported using the web for the purpose of improving performance, quality, and operations. 9CBC85D8

FORMATIVE AND SUMMATIVE EVALUATION Formative Measures - ISSLC Quarterly Assessment - Mid Quarter Assessment on Knowledge and Skills - Knowledge and Skills on LiveText Summative Measures - Professional Growth Plan - Administrators Certification Follow-up Survey - Intern Meeting Evaluation Principals and Program Administrators - Professional Growth Plan Data are regularly and systematically collected, compiled, summarized, analyzed and publicly reported using the web for the purpose of improving performance, quality, and operations. - Thesis, Project, Exam - Intern Meeting Evaluation

FAIRNESS, CONSISTENCY, AND ACCURACY OF MEASURES EVALUATION Inter-rater Summaries Disposition Validation Study Courses ArtifactsRubrics Rubrics and Grades Analysis Statistical Evidence of Validity and Reliability WEST- B WEST -E EBI Principal EBI Teacher Content Validity The extent to which measurement content represents the academic discipline or domain being taught. GPA and WEST E Pass Rates Analysis Standards The unit regularly searches for stronger relationships in evaluations, revising both the underlying data systems and analytic techniques. The regular questioning of validity has led to modifications and changes in assessment technique as well as technology. Aggregated Data for Standards Analysis

THE PROFESSIONAL CULTURE OF ACCOUNTABILITY Web-based Presentation of Data Program Reports Unit Reports Public Survey Assessment Evaluation Changes the Academic Conversation Data are shared with faculty and regularly reviewed by PEABs. The underlying data systems and analytic techniques are evaluated and revised. Recommendations for improvement are based on data-based recommendations from faculty, CTL governance committees, PEABs, and the public. Accountability has become a regular part of faculty conversation.

Assessment System QUESTIONS?