1. Evidence for Impact on Student Learning Tony Norman Associate Dean, CEBS In P. R. Denner (Chair), Evidence for Impact on Student Learning from the Renaissance Teacher Work Sample Assessment. Symposium presented at the annual meeting of the American Association for Colleges of Teacher Education, New York, NY, February 2008. Roger Pankratz Assistant to the Dean, CEBS

2. 2 WKU Participants Fifty-two TWS from two semesters (fall 2006 and 2007) selected to maximize variability in holistic scores (4 – Exemplary, 3 – Proficient, 2 – Developing, 1 – Beginning) 73% Elementary Education Majors 27% Middle Grades/Secondary Education Majors

3. 3 Teacher Work Sample Scores The WKU TWS uses an analytic scoring rubric with 7 targeted standards that are assessed using multiple indicators. The indicators are scored on a 3-point scale: 1 = Indicator Not Met 2 = Indicator Partially Met 3 = Indicator Met 32 to 96. Summing across indicators, the total TWS scores can range from 32 to 96.

4. 4 Learning Impact Measures Percent of students reaching “mastery” on each learning goal. 1.Average Percent Reaching Mastery 2.Percent Mastering Learning Goal 1 3.Percent Mastering Learning Goal 2 The candidates chose the learning goals and the difficulty of mastery level. Learning Goals had to fit the approved curriculum and align with Kentucky Core Content standards. However, the candidates set their own criteria for success on each learning goal.

5. 5 Learning Impact Measures Percent of students showing improvement on each learning goal. 1.Average Percent Showing Improvement 2.Percent Showing Improvement on Learning Goal 1 3.Percent Showing Improvement on Learning Goal 2 Note: Candidates are encouraged to set Learning Goal 2 at a “higher level” in Bloom’s Taxonomy. Many of these are measured by candidate developed rubrics.

6. 6 Reported Impacts on Student Learning

7. 7 Regression Analyses for TWS Scores on the Reported Percent of Students Achieving Learning Goals

8. 8 Regression Analyses for TWS Scores on the Reported Percent of Students Showing Improvement

9. 9 A Closer Look at Key TWS Components 22 fall 2006 TWS were examined for quality of assessments 30 fall 2007 TWS were examines for quality of assessments and quality of unit learning objectives

10. 10 Criteria for Judging Quality of TWS Assessment Alignment with unit objectives Addresses depth of knowledge (DOK) level Sufficient number of items Rubrics provide instructions for judgment

11. 11 Criteria for Judging Quality of TWS Learning Objectives Aligned with Kentucky Content Standards Student friendly Scope – contribution to academic year Curriculum Map Measurable

12. 12 Findings (1) Summary of Holistic Scores of Quality of Unit Objectives Compared to Holistic TWS Scores N=32 Teachers Unit Objectives Scored Higher Unit Objectives Scored Same Unit Objectives Scored Lower 4 (13%)18 (56%)10 (31%)

13. 13 Findings (2) Summary of Holistic Scores of Quality of Unit Summative Assessments Compared to Holistic TWS Scores N=52 Teachers Unit Assessments Scored Higher Unit Assessments Scored Same Unit Assessments Scored Lower 8 (15%)25 (48%)19 (37%)

14. 14 Where WKU Teacher Candidates Appear to Need Improved instruction Writing clear, student friendly unit objectives Writing unit objectives for designated depth of knowledge (DOK) levels Developing classroom student assessments most appropriate for unit objectives Developing classroom student assessments aligned with designated DOK levels

15. 15 For a Copy of the Presentation… http://edtech.wku.edu/rtwsc/publications- and-research.htm