1. Local Evaluation Protocols
Prepared by Joan Lesh
edTPA Coordinator, University of Washington, Seattle
TPAC Implementation Conference Nov. 2012

2. Goals of Workshop Propose a model for local evaluation that includes…
Sharing a brief history and overview of edTPA
Introducing edTPA handbooks, including tasks and prompts
Building a deep understanding of key ideas in edTPA rubrics
Providing time for formative applications and bridging to practice

3. What is the purpose of local evaluation?
To provide guiding formative feedback to teacher candidates as they do edTPA-like tasks (embedded signature assessments, etc.) and the edTPA
To impact student learning in the classrooms of our partner schools
To generate opportunities for collegial conversations that build collective knowledge around key concepts from edTPA and ways to support teacher candidates
To obtain data for program improvement and research

4. Who might use local evaluation?
States and IHE’s not yet in full implementation of edTPA
IHE’s desiring a deeper understanding of edTPA constructs and rubrics
Programs curious to know how their teacher candidates are taking up edTPA “big ideas”
Note use in North Carolina, Wisconsin
Also mention UW local evaluation and WA trainings

5. Who are some key “players” to get involved?
Program Deans and Department Chairs
Faculty at all levels, including adjuncts and teaching assistants
Field Supervisors
Cooperating teachers and partner school administrators
Retired educators

6. Parking Lot Issues Keep in mind today’s goal(s)
Some questions that arise need more time and discussion or that cannot be answered in this session
Use the Parking Lot poster paper (Issues, Concerns, Questions, Other) for questions that arise that go beyond the rubrics or that address policies

7. edTPA Overview

8. What is the edTPA? The Cocktail Party Definition
The Teacher Performance Assessment (edTPA) is a student centered, subject specific, multiple measure assessment of teaching. It is designed to be educative and predictive of effective teaching and student learning.

9. What is the Purpose of the edTPA?
Assess teacher candidates’ abilities to…
Consider academic contexts and students’ prior academic learning and personal/cultural/community assets in planning
Plan instruction based on a clear central focus for student learning (standards/objectives)
Use what they know about the varied needs and assets of learners to adapt instruction
Use instructional practices that support student learning
Ensure that students are engaged in the learning process

10. edTPA as Part of a Multiple Measures Assessment System
Embedded Signature Assessments (campus designed examples)
TPAC Capstone
Assessment
Integration of:
Planning
Instruction
Assessment
Analysis of Teaching
Academic Language
Child Case Studies
Analyses of Student Learning
Analyses of Curriculum/Teaching
Myth – does not replace supervisory evaluations - So the edTPAitself draws on multiple measures AND is designed to serve as a capstone event WITHIN a system of other measures. The other assessments, which are described as Embedded Signature Assessments, may include a number of different kinds of assignments, and vary across programs depending on specific program goals or values.
The next slide unpacks the capstone assessment…
Observation/Supervisory
Evaluation & Feedback 10

11. Like cogs in a wheel

12. Pearson –National Evaluation Systems
Serves as Stanford’s operational partner
Supports Stanford and AACTE with assessment development processes and technical review
Enables scale up and sustainability 12 12

13. Handbook for edTPA

14. Structure of Handbook Tasks Parts to Each Task Rubrics for Each Task
Planning, Instruction, Assessment
Parts to Each Task
What Do I Need to Think About
What Do I Need to Do
What Do I Need to Write
How Will the Evidence of My Practice be Assessed
Rubrics for Each Task
5 Planning + 5 Instruction +5 Assessment = 15 Rubrics TOTAL
+3 Student Voice in WA = 18 RUBRICS TOTAL

15. Remainder of the Handbook
Professional Responsibilities
Context for Learning
Evidence Charts for Each Task
Includes Artifacts and Commentary Specifications
What to Submit
Supported File Types
Number of Files
Response Length
Additional Information
Glossary
“Mouse over” throughout handbook

16. edTPA Task Overview
This example uses Task 2 Instruction: Instructing and Engaging Students in Learning
What to Do
What to Submit
Evaluation Rubrics
Obtain required permissions for videorecording from parents/guardians of your students and other adults appearing in the video.
Identify lessons to videorecord. You should be interacting with students to support them to independently apply the literacy strategy to comprehend or compose text.
Videorecord your teaching and select 1 or 2 video clips (no more than 15 minutes total).
Analyze your teaching and your students’ learning in the video clip(s) by responding to commentary prompts.
Part A: Video Clip(s)
Part B: Instruction Commentary
Instruction Rubrics
Rubric 6: Learning Environment
Rubric 7: Engaging Students in Learning
Rubric 8: Deepening Student Learning
Rubric 9: Subject-Specific Pedagogy
Rubric 10: Analyzing Teaching Effectiveness
Task from El Lit…
***NOTE: video clips = 2 clips no more than 15 min total for El Lit, El Math, MATH
10 min each, thus 20 min total for ELA, Sci, SS

17. Task 1: Planning for Instruction and Assessment
What to Think About
Consider what the task asks you to think about or to do.
Example:
 What do your students know, what can they do, and what are they learning to do?
What Do I Need to Do?
Each box contains directions to follow. Note the verbs.
 Analyze language demands. Select a language function, a key learning task, and additional language demands required by the task.
What Do I Need to Write?
The Commentary Prompts are written with a BOLD-FACED heading.
 1. Central Focus
a. Describe the central focus and purpose for the content you will teach in this learning segment.
Show Handbook simultaneously to illustrate

18. Structure of Rubrics Five Levels:
Level 1 – Struggling candidate, not ready to teach
Level 2 – Some skills but more practice needed to be teacher of record
Level 3 – Acceptable level to begin teaching
Level 4 – Solid foundation of knowledge and skills
Level 5 – Stellar candidate
Language of rubrics is worthy of study:
Reflects valued work in teaching
Has implications for teacher preparation

19. Task 1 Planning Rubrics
Each rubric has a title or focus. Often this corresponds to the Commentary Prompt heading.
A guiding question follows. This should look quite similar to the Commentary prompts.
Rubrics have 5 levels. Most rubrics have only 1 criteria.
Rubrics follow immediately after each set of commentary prompts

20. Task name: Rubric Focus/Title
Rubric Blueprint
Task name: Rubric Focus/Title
Guiding Question
Level 1
Level 2
Level 3
Level 4
Level 5
Struggling candidate, not ready to teach
Some skill but needs more practice to be teacher-of-record
Acceptable level to begin teaching
Solid foundation of knowledge and skills
Stellar candidate

21. Rubric 5: Planning Assessments to Monitor and Support Student Learning
Rubric Focus/Title
How are the informal and formal assessments selected or designed to monitor students’ progress toward the standards/targets?
Rubric Guiding Question
Level 1
Level 2
Level 3
Level 4
Level 5
The assessments only provide evidence of students' literal comprehension of text.
Assessment adaptations required by IEP or 504 plans are NOT made.
The assessments provide limited evidence to monitor students’ abilities to construct meaning from, interpret, and/or respond to complex text during the learning segment.
 Assessment adaptations required by IEP or 504 plans are made.
The assessments provide specific evidence to monitor students’ abilities to construct meaning from, interpret, and/or respond to a complex text during the learning segment.
Assessment adaptations required by IEP or 504 plans are made.
The assessments provide multiple forms of evidence to monitor students’ abilities to construct meaning from, interpret, and/or respond to complex text throughout the learning segment.
Assessment adaptations required by IEP or 504 plans are made
Level 4 plus
The assessments are strategically designed to allow individuals or groups with specific needs to demonstrate their learning.
Assessments are NOT aligned with the central focus and standards/learning targets for the learning segment
How a rubric is organized-Uses ELA Rubric 5
Note that this rubric has more than 1 criteria; most do not – actually it is an automatic ONE if candidates do not attend to IEP/504 plans. No need for preponderance of evidence for this one.
Rubric Performance Levels or Rubric Criteria

22. Unpacking the edTPA Rubrics

23. Goals
Develop a common understanding of the records of practice, evidence and evaluation criteria for the Teacher Performance Assessment (edTPA)
Develop understanding of the subject matter focus for each edTPA
Develop deep understanding of the rubrics, including key words and ideas that distinguish different levels of performance on the edTPA

24. TPAC Artifacts of Practi
edTPA “Records of Practice”
Planning
Instruction
Assessment
Instructional and social context
Lesson plans and Instructional materials, student assignments
Planning Commentary
Video Clips
Instruction Commentary
Analysis of whole class assessment
Analysis of learning and feedback to THREE students
Assessment Commentary
Analysis of Teaching Effectiveness
Academic Language Development

25. Two Types of Evidence… Teaching Artifacts Commentaries Lesson plans
Instructional materials
Assessment tools and evaluation criteria
Video clip(s) of teaching
Analysis of student learning
Student work samples
Planning Commentary
Instruction Commentary
Assessment Commentary

26. Evaluation Criteria
The evidence submitted in the edTPA is evaluated across five components of teaching practice:
Planning
Instruction
Assessment
Analyzing Teaching
Academic Language

27. Targeted Competencies
PLANNING
Planning for content understandings
Planning to support varied student needs
Planning assessments to monitor and support student learning
INSTRUCTION
Demonstrating a positive and engaging learning environment
Engaging students in learning
Deepening student learning during instruction
Subject specific pedagogy
ASSESSMENT
Analyzing student work
Providing feedback to guide learning
Supporting students’ use of feedback
ANALYZING TEACHING
Using knowledge of students to inform planning
Analyzing teaching
Using assessment to inform instruction
ACADEMIC LANGUAGE
Identifying and supporting language demands
Evidence of language use to support content understandings
Each of these corresponds to a rubric and at least one commentary prompt

28. Central Focus for All Rubrics
Each discipline has key understanding/skills
Key understanding/skills go well beyond literal comprehension, simple application, or recall of knowledge/facts
Candidates must show that they are helping students achieve the key understanding/skills of the specific discipline
This is a central idea for the TPA – that candidates are teaching more than facts and low level skills. Candidates should be demonstrating that they are teaching students disciplinary concepts and skills that have relevance beyond school.

29. Examples of Key Understanding/Skills
Subject Area
Key Understanding/Skills
English-Language Arts
Create a written product demonstrating comprehension, construction of meaning from, and interpretation of complex text
History-Social Studies
Use facts, concepts and interpretations to make and explain claims and arguments about a significant historical event, theme, or social studies phenomenon
Science
Use scientific concepts and scientific inquiry to investigate and explain a real world phenomenon
Mathematics
Demonstrate conceptual understanding, procedural fluency, and mathematical reasoning/problem solving skills
This slide identifies the ‘central focus’ for four subject area disciplines. Note that several of the rubrics draw specific attention to the key understanding/skill in the subject area. Even those that do not explicitly name the central focuses require attention to them. For example, for candidates to adequately address academic language, they have to know what the central focus is and make sure that the reading and writing skills as well as the vocabulary relevant to the central focus are attended to in planning, instruction, and assessment.

30. Rubric 9---Subject Specific Pedagogy
Elementary Literacy=How does the candidate support students to apply the essential literacy strategy?
ELA =How does the candidate use textual references to help students understand how to construct meaning from and interpret a complex text?
Mathematics =How does the candidate use representations to develop students’ understanding of mathematical concepts and procedures?
Science =How does the candidate facilitate students’ analysis of the data based on scientific inquiry?
History/Social Studies = How does the candidate support students in using evidence from history/social studies sources to build and support arguments?
This rubric language varies for content areas…see red highlights for differences

31. Rubric Progression Expanding repertoire of skills & strategies Deepening of rationale and reflection
Not Ready
Early Novice
Highly Accomplished
Beginner 1 5
Teacher Focus
Student Focus
Whole Class
Individuals/ Flex. Groups
Integrated,
Intentional & Well Executed
Fragmented, Indiscriminate

32. Process for Understanding the Rubrics
Individually read the rubrics carefully
As a small group:
Identify the key concept(s) the rubric is measuring
Identify the language that distinguishes one level from the next higher level; Start with Level 3
Identify characteristics of ‘automatic 1’
Discuss examples for each level you have seen in your own teaching and your observations of other teachers
Debrief with the whole group

33. Just a Reminder… Please READ before you DISCUSS!
Just a reminder. Folks often jump in with discussion before everyone has had time to read. We recommend that participants read rubrics in sets and then begin discussion.

34. Debriefing Questions Focus on each rubric individually
Key Elements
What language is KEY to…
A Level 3 score
A score BELOW Level 3
A score ABOVE Level 3
What will automatically lead to a score of 1?
Your examples

35. Discussing Key Elements of the Rubrics
OPTION A
Create whole group charts of Key Elements To model process
For whole group work OR
OPTION B
Create small group charts of Key Elements To compare/contrast
To jigsaw, gallery walk

36. Debriefing---OPTION A
(Whole Group, Leading Participants):
One person can guide debrief while another records their thoughts about key words/key ideas using flip chart paper.
Prepare one sheet of chart paper for each rubric. Set up with the rubric number and question at the top (or the number and an abbreviated title).
Write Level 3 in the middle with NOT YET 3 and BEYOND 3, leaving space for key words/ideas. It is also helpful to use different colors for each level to draw attention to the different levels of performance.
Option A: Use this option if you will lead participants in capturing their thoughts about key words/ideas. It’s easiest if one person is guiding the debrief while another person writes the key words/ideas on flip chart paper.
Move quickly and, where necessary, slightly adjust the wording to focus on key words/ideas OR note key words/ideas that the participants miss.

37. Debriefing---OPTION B
(Small Groups, Sharing with whole):
Each group has own chart paper to record.
Can be done as a jigsaw, with each group sharing out about a different rubric or as a gallery walk
Can also be done as a compare/contrast activity to show similarities and differences between group interpretations of rubrics
Option B: Use this option if you will have participants complete the flip charts themselves.
Provide one sheet of chart paper for each rubric. Set up with the rubric number and question at the top (or the number and an abbreviated title) and Level 3 in the middle with BELOW 3 and ABOVE 3 with space for key words/ideas. It is also helpful to use different colors for each level to draw attention to the different levels of performance.
Give each table a flip chart page for each rubric and have them work through the rubrics themselves. Then have groups compare across groups using a ‘gallery walk’ or share using a jigsaw format.

38. Sample Key Elements Chart for Rubric 1
Planning 1: Planning for Content Understandings
Level 1
Not aligned; No connections
Content inaccuracies
Not Yet Level 3
Inconsistent alignment
Vague connections
Level 3
Aligned; Plans build on each other
Connected lessons
“Do it”
Beyond Level 3
AND
Consistent alignment; Plans build to support deep meaning
Clear, meaningful connections
Illustrating reasoning

39. A Practice Opportunity
1. As a individual or with a partner, PRACTICE, using Planning Rubric 2.
2. Focus on the Key Elements for each rubric level
Identify what language is KEY to…
A Level 3 score
A score BELOW Level 3
A score ABOVE Level 3
3. What will automatically lead to a score of 1?
4. Consider examples from your own experience that might map to these different levels.

40. Key Elements Chart for Rubric 2
Planning 2: Planning to Support Varied Student Needs
Level 1
Not Yet Level 3
Level 3
Beyond Level 3
Practice w/Rubric 2 and then another Planning Rubric
Reminder to start at Level 3

41. Key Elements Chart for Rubric 2
Planning 2: Planning to Support Varied Student Needs
Level 1
No evidence of planned supports
No consideration of differences in learners
“For their dolls”
Not Yet Level 3
Supports loosely tied to objectives
504’s & IEP’s are addressed
Level 3
Supports tied to objectives
Whole class focus
504’s and IEP/s are addressed
Beyond Level 3
Individual or small group focus
Examining group characteristics
Addresses groups w/similar needs
NC Key Elements---compare w/own
Reminder to start at Level 3

42. Key Elements Chart for Rubric #___
Planning #__: _____________________________________
Level 1
Not Yet Level 3
Level 3
Beyond Level 3
Follows Practice w/Rubric 2 – se Planning Rubric #4
Reminder to start at Level 3

43. Key Elements Chart for Rubric #4
Planning #4 : Identifying and Supporting Language Demands
Level 1
Support is missing or not aligned w/ lang demands
Inconsistent alignment of lang demands with lang function
Not Yet Level 3
Vocab is identified as major lang demand
Definitional vocabulary
Basic, Memorize
Level 3
Vocab and additional lang demands identified
Plans include general support
Whole class
Beyond Level 3
Plans include targeted support
Multiple levels of opportunities to use lang based on individual needs
Differentiation
Error analysis
Application
Planning Rubric 4---NC Sample
Reminder to start at Level 3

44. Other Activities to Get to Know the Rubrics
Put rubric criteria on strips
Place in sequential order, then check with rubric
Match rubric criteria to rubric title/guiding question
Other ideas?

45. Consider all of the edTPA Rubrics
Patterns in Rubrics
Consider all of the edTPA Rubrics
What patterns or trends do you see?
What are some indicators of a Level 1 score?
What about a Level 3?
What do you notice about Levels 4 and 5?

46. Closure
Review the key words/ideas you have captured
Think about your work with pre-service teachers and answer these questions:
What aspects of your program (curriculum, instruction, assignments, supervision, etc.) prepares candidates to do this work?
What next steps do you need to take to prepare candidates to be successful on the edTPA?
This slide can be ‘take away’ questions or you can have groups discuss these questions within program before they leave.

47. Formative Applications

48. INQUIRY

49. How does the edTPA align with your programs?
To what extent does it reflect program values?
Where do you predict that students would do well? Where might they struggle?
What core values and program emphases are not captured in the edTPA?
What kinds of assessments do you currently use (or might you need) to get at these?
What else do you consider critical? edTPA does not measure everything

50. What similarities do you find?
Where are some places in your own program/curriculum where candidates are already doing edTPA-like tasks?
How does this align with your own courses, assignments, assessments, and field work experiences?
How could you apply edTPA-like ideas to your own programs right now?

51. “Cultures of Evidence”
Critical and collegial conversations about edTPA adoption
Inquiry and program improvement (not compliance) as motivational orientation
Strategic inclusion of faculty in examining cases of candidate performance at regularly scheduled events
Affirmation of program values and identity
Deepens candidate understanding and practice
From Peck and McDonald’s Study of PACT Implementation (2011)
Survey of all 32 PACT institutions, interviews with about half and 3 case studies.

52. Bridge to Practice
How do you envision you could apply these rubrics in a variety of contexts?
What are some places in your program where similar rubrics are already in place?

53. Thank you! jlesh@u.washington.edu
General Information
General Questions
Join TPAC Online