1. Aligning Academic Review and Performance Evaluation (AARPE)
Virginia Department of Education
Office of School Improvement
Aligning Academic Review and Performance Evaluation (AARPE)
Important Facilitator Notes:
Information in quotations indicates scripted material
Information in parentheses indicates facilitator directions/information
___________________________________________
(DECISIONS TO BE MADE PRIOR TO SESSION 2: Who and how will you address Next Steps?; Where did you end up with Standard 3 – this will determine how you modify this slide deck to pick up during Session 2; Will you incorporate work on the Principal standards or not – this will determine how you modify this slide deck; How will you address participants who do not bring the requested evaluation feedback documents; How will your LTPs be incorporated if at all?)
“Good morning. Has everyone signed in and gotten a name tag and a handout packet? Take a quick minute to make sure that you are seated at the correct table. See labels on each table.”
(Introduce facilitators and quick introductions around the room.)
“This will be a full day as we will catch up on those pieces of the work that we did not complete last time as well as introduce new material.”
HISTORY (In case this needs to be reviewed for new participants):
“Aligning Academic Review and Performance Evaluation AARPE: History of this professional development: began last year ( ) with differentiated technical assistance requested by some divisions. As a result of feedback on a more differentiated delivery, steps were put in place to begin development of AARPE. Academic reviews were conducted in all divisions across the state. From those, some general findings were alignment of lesson planning, lesson observation, leadership feedback and monitoring, and outcome based professional development. Schools/divisions in this room followed the same general findings.
In further discussion, the PD grew to be more comprehensive and to weave priority and academic review needs into the technical assistance for this year. So, then, how were you selected?
All Priority School (federal sanction) Principals;
Others – Selected because you provide direct support to schools in the room and you will be training all other 2nd and 3rd year warned and accreditation denied school principals with these materials.
The tone of the training is very important. We want this to be responsive and adaptive to the needs of your district/school. We plan to model this throughout the training.
To that end, let’s talk about what we already know about working with you. We know that you genuinely want to improve and want PD to be connected to the real work of the job. One message we heard is the overload that comes with being a priority or struggling school. We hope to work as a team in this room to determine prioritized goals and implement them with fidelity. We want the work to be important and meaningful for you and your students. That is why we will use your actual work artifacts- the forms that you were asked to bring to this training. As we do this work, we need to acknowledge that the work may be messy and at times painful. We are not here to evaluate but you are here to evaluate your own strengths and weaknesses and to identify areas for improvement. There are many different hats in the room; during these sessions it is not the intent for anyone to wear the evaluator hat.
You’ll see that our first few slides are the overview and purpose. We don’t want this to be overwhelming but we do want to give you the big picture. So as we go quickly through this, remember we will break it out into small chunks as we move forward. Note that all of the slides are in your handout packet. TAKE A FEW MINUTES TO ORGANIZE AND SEPARATE HANDOUTS (slides). The packet is on 3-holed paper because we are asking you to set up an AARPE 3 ring binder so you can add training materials each month and bring the notebook each month.” 
Session 2
Technical Assistance Sessions for Selected Divisions

2. Technical Assistance Initially Offered 2014-15 1 of 5
Overview and Purpose
To improve instruction and instructional leadership practices by strengthening the alignment between the Performance Standards for Teachers and Principals and the Lesson Planning, Lesson Observation, Professional Development, and Leadership Academic Review Tools.
To develop sample evidence for the sample performance indicators in Teacher Performance Standards for professional knowledge, instructional delivery (planning), assessment of and for learning, and learning environment. The sample evidence for each performance indicator will become a tool that can enhance the division’s observation tools. (District and School Leadership)
(Review the two bullets, shortening as appropriate given the previous review in Session 1.)
“As we discussed previously, we will be working with our own forms for walkthroughs, observations, and other feedback, etc. This work is not about changing a form.
Regarding Bullet 2: It is our intent that the sample evidence we create during these sessions will support our forms and processes. We will develop the tool so that we can have it in hand for use. The goal is that the same tool(s) and language at the classroom, school, and district levels are defined.
Also, we will look at the connections between academic review tools and performance indicators. You will gain a strong understanding of how these two documents are aligned.
Finally, we will practice the use of our tools in both this setting and in the school setting. I’ll explain more about that at the end of day two. We will answer the questions: When and how do you provide feedback? How do you follow-up? What would help you?”

3. Technical Assistance Initially Offered 2014-15 Overview and Purpose 2 of 5
Overview and Purpose (continued)
To develop sample evidence for the sample performance indicators in the Principal Performance Standard for instructional leadership. The sample evidence for each performance indicator will become a tool that can enhance the division’s tools. (District Leadership)
To enhance feedback skills of building and district leaders.
(Review bullets.)

4. Technical Assistance Initially Offered 2014-15 3 of 5
Outcomes
Each participant will
Set clear and measurable goals for his/her work.
Implement action steps with fidelity.
Monitor and provide feedback.
Adjust based upon monitoring and data.
Repeat the cycle.
“It is always our intention to model what we teach. One important component is to set and abide by specific outcomes for our time together. We’ve established these outcomes for today.”
(Review each bullet)
“Are there any others we should consider?”
(If any additional are agreed upon, make note of it directly on the slide.)

5. Technical Assistance Initially Offered 2014-15 4 of 5
Roles/responsibilities
Principals/district staff will use their own work as a starting point and will bring a predetermined number of completed walkthrough feedback forms and formal observation documents to each session throughout the year.
Principals, appropriate district staff, and OSI-approved personnel will conduct monthly walkthroughs and formal observations per the established timeline.
District staff will support and monitor principals’ delivery of professional development on the sets of sample evidence developed to appropriate school staff.
Lead Turnaround Partner (LTP) will support AARPE training in their work with building leaders and other staff.
“It’s prudent to take a few minutes to review our Roles/Responsibilities regarding this technical assistance.”
(Review each bullet adding the following statements. Modify based on your audience and the need for review given the previous review in Session 1.)
“VDOE training materials for each session will be provided to each principal from the district office.
We will come to consensus on this number today. Once determined we will consider this a training non-negotiable for all participants.
Beginning after the session 2 training, one day will be identified for the principal and designated district leader to conduct walkthroughs and observations for inter-rater reliability. Logistics of this will be worked out at the division level. Work completed on this day will count toward the number identified in bullet 2. A calendar will be developed for each school.
Through scheduling and coaching, district staff will support the principal as they conduct professional learning with school staff. A calendar will be developed for this and district staff will attend.
The key is that we are all speaking the same language and using the same tools where appropriate. All of our work by school and district should be tied to the academic review tools and performance standards.
Any questions before we proceed?”

6. Technical Assistance Initially Offered 2014-15 5 of 5
4. Professional Development Norms
Arrive on time and stay the entire session.
Be an active participant. (Complete in-session and between-session work.)
Reserve school/district/personal business for established breaks.
Maintain confidentiality.
Use large sticky notes to indicate additional professional development needed/desired.
(Adjust/modify this slide to meet district expectations. Some suggested remarks follow.)
“We know how challenging it is for you to be away from your school and how easy it can be – especially with technology – to get caught up in what’s going on there instead of being laser focused on the learning going on here. As is the case with any learning experience, setting norms for the day will help ensure a learning environment that is effective for each of you. To that end, let’s review the list.”
(Proceed to review all five bullets, expanding on points as necessary.)
“One additional note - you’ll see that we’ve established a Parking Lot (indicate the chart paper on the wall labeled as such). Throughout our two days together, this is the place to note any lingering questions, especially those you’d like answered now but don’t want to ask aloud to the group. It’s also a place to note any additional professional learning needs you discover as we proceed. Use your post it notes from the table, and please post in real time. Sometimes when we wait, we forget. If you are absolutely reluctant to post here, feel free to your facilitator and he/she will post the question anonymously for the group”

7. Essential Questions
How do the components and criteria of the selected academic review tools align with the Teacher and Principal Performance Evaluation Standards lesson observation tool?
How do we use these tools for school improvement?
How do we know we are making progress?
What are the next steps?
“Virginia’s academic review focused on the alignment of the written, taught, and tested curriculum with tools and supporting resources being developed by Strong and Associates. For the AARPE Part 1 (initially offered in ) Academic Review/ Priority technical assistance, we are focusing on the Lesson Planning, Lesson Observations, Leadership, and Professional Development tools as well as selected teacher and principal performance standards. Our work will be around these four essential questions.”
(Read Question 1) “We looked at this at session one. You’ll recall the activity where you went through the steps of specifically identifying how the two tools aligned. We will continue to keep this at the forefront as we proceed with our work.”
(Read Questions 2 – 4.)
“We will continue to review the performance standards as we work on developing the sample Look Fors/evidence.”

8. Next Steps for AARPE 2 (Completed by the date of Session 2)
District Next Steps:
(Slide 8 represents Next Steps for divisions. The facilitator will add slides by division and then division staff will determine how best to share the information so that it is consistent and clear. These Next Steps were identified at the end of Session 1. This slide should be modified by the facilitator prior to the start of Session 2.)

9. Main Areas for Evidence Collection during Observations Where are we?
Standard 1: Professional Knowledge Standard 2: Instructional Planning Standard 3: Instructional Delivery Standard 4: Assessment of/for Learning (formative assessment) Standard 5: Learning Environment Standard 6: Professionalism Standard 7: Student Academic Progress
“This is our list of all seven standards. The standard in green is one for which you have completed the vetting process and which has been finalized to-date. The standards in blue are the ones that we will continue to refine through this technical assistance. Today we will continue our work on Standard 3 for teachers and we will revisit Standard 1 for principals so that we can finalize those Look For lists.

10. Enhancing Teacher Quality: Questioning
Standard 3: Instructional Delivery The teacher effectively engages students in learning by using a variety of instructional strategies in order to meet individual learning needs.
Sample Performance Indicators:
3.1 Engages and maintains students in active learning.
3.2 Builds upon students’ existing knowledge and skills.
3.3 Differentiates instruction to meet the students’ needs.
3.4 Reinforces learning goals consistently throughout the lesson.
3.5 Uses a variety of effective instructional strategies and resources.
3.6 Uses instructional technology to enhance student learning.
3.7 Communicates clearly and checks for understanding.
“You’ll recall that we completed steps 1-2 of the Look For activity for Teacher Standard 3 during session 1. As a reminder, here are the Sample Indicators for Standard 3. In this standard, the teacher is engaging students in active learning, building upon prior knowledge and skills, differentiating instruction, reinforcing learning goals, using a variety of strategies and technology, and once again, communicating clearly and checking for understanding.”
TQR Teacher Quality Resources, LLC (c) 2005

11. Teacher Standard 3 – Look Fors Activity
Step One: Individual Work - Look through your observations, walkthroughs, and/or written feedback and highlight and label (3.1, 3.2, etc.) any examples of “Look Fors” for Performance Standard 3 Sample Performance Indicators.
Step Two: Table Groups – Combine your list of “Look Fors” into one unduplicated list and have one person from your table the list to facilitator. Include the names of all participants at your table at the bottom of your list.
Step Three: Facilitators share combined lists and assign one indicator to each table. Table group discusses examples and determines a final list of sample “Look Fors” for the specific Performance Standard 3 Sample Performance Indicator assigned to their table. Assign a Recorder to type the list in an .
Step Four: Table groups report out (Whole Group Feedback). Assign a Reporter to present the list. While the Reporter shares out, the Recorder makes all changes to the list in the and sends the final list to (INSERT )
“During our previous work, you have used your own documents to establish a list of Look Fors for Teacher Standard 3, Instructional Delivery. We concluded with table group work to establish a final draft list.”
(DISTRIBUTE LISTS)
“We are going to take some time to re-visit Step 2 and review these lists at table groups in order to confirm that the lists contain clearly stated Look Fors that meet our established criteria for evidence.
Review Look- Fors for all indicators. Be cognizant that your statements must be concise and specific- this is not the place for generalities.  In addition to the criteria we displayed earlier, ask yourselves these questions:
Is each statement specific?
Is each statement concise and observable (What does it look like in the classroom)?
Have we avoided generalities?
Do we have Look- Fors that address both teacher and student behavior?
Do we have Look -Fors that address what the indicator does look like as well as what it does not look like?”
(Circulate and OBSERVE only. Watch time and extend time if necessary. Prior to starting Step 3, assign one indicator to each Table group for Step 3.)
“Now it's time to return to step 3. Take the list of Look Fors that were created and vet the list for the specific indicator that your table is assigned, removing any the group deems to not be Look Fors – again note the criteria displayed in the front.  Take the time to review each Look For and make sure you agree that it meets the criteria. Please be thorough. We will monitor to keep track of time and when groups are finished with their sample performance indicator, one member should the final list to (ENTER PRESENTER NAME HERE) and another member should be prepared to share the final list with the entire group. At that time, all groups will have a chance to ask questions or offer suggestions for the final version.”
 (Circulate and OBSERVE only. Watch time and adjust to shorten or lengthen as necessary.)
 “We've received all lists and will begin the process of reporting out to the entire group.”
(The list is displayed but it’s the Group recorders who make changes to the list as they are proposed and agreed upon during the whole group presentation. When the group is done, the recorder sends the with the final version of the list to the PRESENTER.)
“You will receive a copy of the Teacher Standard 3 Look Fors when it has been compiled.”

12. Standard 1: Instructional Leadership (Principal) 1 of 2
Sample Performance Indicators:
1.1 Leads the collaborative development and sustainment of a compelling shared vision for educational improvement and works collaboratively with staff, students, parents, and other stakeholders to develop a mission and programs consistent with the division’s strategic plan.
1.2 Collaboratively plans, implements, supports, monitors, and evaluates instructional programs that enhance teaching and student academic progress, and lead to school improvement.
1.3 Analyzes current academic achievement data and instructional strategies to make appropriate educational decisions to improve classroom instruction, increase student achievement, and improve overall school effectiveness.
1.4 Possesses knowledge of research-based instructional best practices in the classroom.
1.5 Works collaboratively with staff to identify student needs and to design, revise, and monitor instruction to ensure effective delivery of the required curriculum.
1.6 Provides teachers with resources for the successful implementation of effective instructional strategies. .
“We are familiar with these performance indicators for Principal Standard 1: Instructional Leadership. You have been working on Look Fors especially for those indicators highlighted in Red.”

13. Standard 1: Instructional Leadership (Principal) 2 of 2
Sample Performance Indicators:
1.7 Monitors and evaluates the use of diagnostic, formative, and summative assessment to provide timely and accurate feedback to students and parents, and to inform instructional practices.
1.8 Provides collaborative leadership for the design and implementation of effective and efficient schedules that protect and maximize instructional time.
1.9 Provides the focus for continued learning of all members of the school community.
1.10 Supports professional development and instructional practices that incorporate the use of achievement data and result in increased student progress.
1.11Participates in professional development alongside teachers when instructional strategies are being taught for future implementation.
1.12 Demonstrates the importance of professional development by providing adequate time and resources for teachers and staff to participate in professional learning (i.e., peer observation, mentoring, coaching, study groups, learning teams).
1.13 Evaluates the impact professional development has on the staff/school improvement and student academic progress.
“We are familiar with these performance indicators for Principal Standard 1: Instructional Leadership. You have been working on Look Fors especially for those indicators highlighted in red.”

14. Principal Standards Look Fors Activity
Step One: Individual Work - Look through your observations, walkthroughs, and/or written feedback and highlight and label any examples of “Look Fors” for the appropriate principal performance standards.
Step Two: Table Groups – Combine your list of “Look Fors” into one unduplicated list and have one person from your table the list to facilitator. Include the names of all participants at your table at the bottom of your list.
Step Three: Facilitators share combined lists and assign one indicator to each table. Table group discusses examples and determines a final list of sample “Look Fors” for the specific Performance Standard 1 Sample Performance Indicator assigned to their table. Assign a Recorder to type the list in an .
Step Four: Table groups report out (Whole Group Feedback). Assign a Reporter to present the list. While the Reporter shares out, the Recorder makes all changes to the list in the and sends the final list to (INSERT )
“During our previous work, division leaders have used their own documents to establish a list of Look Fors for Principal Standard 1, Instructional Leadership. We concluded with table group work to establish a final draft list. Our next step will be to have the entire group vet the initial set of Look Fors. This ensures that both division staff and principals are involved with the process.”
(DISTRIBUTE LISTS)
“We are going to take some time to review these lists at mixed table groups in order to confirm that the list contains clearly stated Look Fors that meet our established criteria for evidence. Review Look Fors for all indicators. Be cognizant that your statements must be concise and specific- this is not the place for generalities.  In addition to the criteria we displayed earlier, ask yourselves these questions:
Is each statement specific?
Is each statement concise and observable?
Have we avoided generalities?
Do we have Look Fors that address both teacher and student behavior?
Do we have Look Fors that address what the indicator does look like as well as what it does not look like?”
(Circulate and OBSERVE only. Watch time and extend time if necessary. Prior to starting Step 3, assign one indicator to each Table group for Step 3.)   
“Now it's time to return to step 3. Take the list of Look Fors that were created and vet the list, specifically for the indicator to which your Table has been assigned, removing any the group deems to not be Look Fors – again note the criteria displayed in the front.  Take the time to review each Look For and make sure you agree that it meets the criteria. Please be thorough. We will monitor to keep track of time and when groups are finished with their sample performance indicator, one member should the final list to (ENTER PRESENTER NAME HERE) and another member should be prepared to share the final list with the entire group. At that time, all groups will have a chance to ask questions or offer suggestions for the final version.”
(Circulate and OBSERVE only. Watch time and adjust to shorten or lengthen as necessary.)
“We've received all lists and will begin the process of reporting out to the entire group.”
(The list is displayed but it’s the Group recorders who make changes to the list as they are proposed and agreed upon during the whole group presentation. When the group is done, the recorder sends the with the final version of the list to the PRESENTER.)
“You will receive a copy of the Principal Standard 1 Look Fors when it has been compiled.”

15. Why Evaluation Only Is Not Enough
Teacher evaluation is not an end in itself, but a means to an end —
TEACHER IMPROVEMENT.
“Before we move on, let’s review again the basis for our learning. While we do observe teachers in order to collect evidence for evaluations, it’s important to remember that teacher professional growth is one of the essential reasons that a teacher evaluation system is designed and implemented. D.R. Davis and colleagues suggest that ‘…school-based administrative and professional leadership play essential roles in determining the meaning and value of teacher evaluation in schools, and how teacher evaluation can extend beyond its ritualistic traditions to improve teaching and learning.’ The leadership makes the difference between ‘perfunctory summative teacher evaluation and meaningful assessment of the teaching and learning process that has the potential to enhance the quality of teaching and student learning.’ Therefore, if we want teacher evaluation to result in improved instruction, we need to acknowledge the essential role that principals, who are the schools’ instructional leaders, play in providing feedback to their teachers. By the same token if we want principal evaluation to result in improved instruction, we need to acknowledge the essential role that district leadership plays in providing feedback to principals.”
Davis, D. R., Ellett, C. D., & Annunziata, J. (2002). Teacher evaluation, leadership and learning organizations. Journal of Personnel Evaluation in Education, 16(4), p. 288.

16. Making Permanent Changes: The Importance of Leadership
Explicit Training and Expectations on Needed Skills Plus
Explicit, Sustained Emphasis and Monitoring of New Skills Equals
Change in Practice
“During the course of the academic reviews, we’ve discussed many processes for unpacking the standards, planning lessons, and using high-yield strategies. The only way that any of these processes will be successful and lead to greater student learning, however, is if we provide our educators with explicit training on these skills and our expectations for how we expect to see them used. You’ll recall that this is addressed in part of the Leadership Tool used for both schools and divisions. Even that is not enough, however. We also must have an explicit, sustained emphasis on these skills which is done through monitoring and feedback at both the division and the school level. Principals at schools should be monitoring teachers on their use of these skills and providing them with feedback; divisions should be monitoring the principals’ feedback to the teachers. We cannot assume that if we introduce teachers once to a process, they will automatically use it forevermore; true permanent change takes time, perseverance, and support from the leadership. You have begun that process with these sessions.”

17. STANDARDS & INDICATORS
Focus of Evidence
(Evidence is the available body of facts or information indicating whether a belief or proposition is true or valid.)
STANDARDS & INDICATORS
Teacher Practice
Student Learning
”In our work on Standards 1 and 3 we have delved into evidence. Let’s go back and look at our working definition. Evidence is the available body of facts or information indicating whether a belief or proposition is true or valid. This definition is important for two reasons. First, evidence is based on a belief or proposition. In this case, the belief or proposition is that the teacher is Proficient in the standards for effective teachers. Evidence, therefore, should be either fact- or information-based and indicate whether that proposition is true. When we think about evidence, we need to remember to frame everything through the lens of the standards and their indicators. Once we have that framework, we then need to concentrate on two specific things: (1) teacher practice and (2) its effect on student learning.
When evaluators do observations in classroom, they should strive to ensure that their evidence:
is standards-based. We’ll refer shortly to the Teacher Performance Evaluation System standards for Virginia in order to determine what this means;
can include both examples of the teacher meeting or not meeting the expectation for the standard;
should not include judgments—in Virginia’s Guidelines for Uniform Performance Standards and Evaluation Criteria for Teachers, it is highly recommended that individual observations are not rated, and that ratings are reserved for summative evaluations only. The reason for this is simple: even if we observe 3-4 classes a year for a single teacher, that’s still less than 1% of overall time the teacher spends instructing students.”

18. Reliability in Observations
Common Reliability Issues in Observations
Lack of criterion reliability
Establishing intra-evaluator reliability
Lack of inter-evaluator reliability
“Now that we have reviewed the basics about collecting evidence, let’s turn our attention to new learning, specifically how we can increase the reliability of our observations and decrease potential biases. How consistent are the evaluator’s observations with those of an expert observer? One problem is a lack of criterion reliability. Criterion reliability deals with how close an evaluator’s evidence collection is to an expert’s evidence collection. Higher consistency with the expert’s evidence means higher criterion reliability. In many cases, criterion reliability is related to how well an evaluator understands the standards and indicators. Another major problem is Intra-Evaluator Reliability, which has to do with how consistent an evaluator is in his or her own observations of teachers. One last problem with reliability is a lack of inter-evaluator reliability, which has to do with the consistency between the evidence collection of two or more evaluators. Let’s look at some solutions to these issues.”

19. Solution: Establishing Criterion Reliability
Training with an expert observer
Practice scoring using videos, documentation, etc.
Instruction on how to interpret evidence
Information about common sources of systemic observer error
“There are many ways to increase criterion reliability. Training with an expert observer includes participants working with an expert trainer to watch videos of lesson observations and use documentation to practice collecting evidence. The trainer provides instruction on both how to collect and interpret the evidence. Training on the common sources of observer error, and how to counteract these errors, also helps increase the criterion reliability.”
The MET Project, 2013

20. Solution: Establishing Intra-Evaluator Reliability (1 of 2)
Watch the same lesson multiple times with time between
Give ratings based on multiple pieces of evidence rather than single observations
“One solution for increasing intra-evaluator reliability is having evaluators understand whether there exists some discrepancy in their own observations by having them watch a videoed lesson more than once, with time between the two viewings. Evaluators compare the evidence they collected during the original viewing of the lesson with the evidence they collect during the second viewing. They then ask themselves: Are there differences? How great are they? Why are they there? This simple experiment can help evaluators become aware of an intra-evaluator reliability issue in order to take steps to fix it. Another way to help counteract this reliability issue is to not have evaluators rate a single observation, but emphasize that ratings should only be made based on multiple pieces of evidence gathered over multiple observations.”
The Met Project, 2013

21. Solution: Establishing Inter-Evaluator Reliability (2 of 2)
Calibrate evidence collection during initial training
Conduct tandem observations and performance reviews with multiple evaluators
“One way to establish inter-evaluator reliability from the beginning is through reliability trainings that calibrate evaluators’ evidence collection during observations. After the initial training, the inter-evaluator reliability should be periodically re- established through tandem observations in which multiple evaluators observe the same lesson, compare the evidence they collected, and how they interpreted it. This leads to our next section: decreasing bias during observations. Observers collect idiosyncratic or irrelevant evidence during lesson observations. To decrease this inconsistency, we will conduct group observations in each of the schools represented here in this technical assistance training.”
The Met Project, 2013

22. Inter-Rater Reliability: In Action
Training Purpose: Why this work?
Training Definitions:
Formal Observation – 30 Minutes
Walkthroughs – 15 minutes
How many Formal Observations per month?
How many walkthroughs per month?
Framework for Inter-Rater Reliability Work
Who is observed?
When does this work begin?
Time Management
“As you’ll recall from earlier, one of the primary purposes of our technical assistance this year is to improve the reliability of our observations and feedback. We will use inter-rater reliability work to do this. We call it inter- rater instead of inter evaluator because WE are not evaluators. Our primary purposes are to: Increase principal’s effectiveness, encourage principal feedback to teachers, and to provide the opportunity to monitor principals.
The state makes the following recommendation:
Observations should last the entire block for formal observations.
In lieu of the entire block, observations should last no less than 45 minutes preferred/30 minutes minimum.
The role of the OSI-approved personnel is to offer additional observer feedback on the performance indicators and to encourage the use of our technical assistance learning.
Ultimately, the observation team is focusing on the question Is it really evidence?
Let’s talk further about numbers 3 and 6.”

23. Framework for Inter-Rater Reliability Work
Who: Principal, AARPE Key Division Leader, OSI-approved personnel
When:
One day between Session 2 date and Session 3 date
One day between Session 3 date and Session 4 date
One day between Session 4 date and Session 5 date
These 3 days are identified by the principal, AARPE Division Leader, and OSI-approved personnel and are considered as non-negotiable.
What: Established Recommendations for each of the 3 days:
4 – 30 minute observations AND
4 – 15 minute walkthroughs
The school division’s forms will be used by all division staff. (Note: OSI-approved personnel will use VDOE sample observation form and the division’s walkthrough form.)
4. Debriefing: Formal debriefing will be conducted at the next scheduled AARPE session.
“As stated earlier, your observation team will include the principal, a member of your division school turnaround team, and your assigned state OSI-approved personnel. You are expected to have a team observation one day for each of the three identified windows of time. As you can see in number three, we are making these recommendations with respect to number of observations, duration, etc. It is critical to note that these observations are counted toward any requirement your division already has established regarding teacher observation so that the work is meaningful and does not add to your work load. We will begin our debriefing of your observations at AARPE session 3.
We are going to take time now to identify the specific dates outlined in number two and the specific number of observations/walkthroughs as well as duration of each.
(At this point the discussion should be turned over to division staff to lead the conversation. Key factors should include the state recommendations as well as the requirements already established by the division. When dates are finalized, the facilitator should add actual session dates to this Slide for future reference.)

24. Time Management
Plan ahead, establish and publish (with appropriate staff) times for observing in classrooms.
Discuss with appropriate district staff, gaining stakeholder support by making it known that there will be times the administrative staff is unavailable while observing instruction in the classroom.
Chunk out time: Mornings on Wednesdays and Fridays, afternoons on Tuesdays, etc.
Plan for distributed leadership: who will be your designee for what? What is an acceptable reason to interrupt?
4. What time management strategies are you planning for this year?
“Time is almost always the key concern raised when we discuss the effective observation and evaluation of teachers. To that end, lets review these recommendations.”
(Review each Specifically emphasize that, with principals especially, it will be important to be clear about the types of emergency only situations that can be reason for disrupting the established observation schedule..)
“Time is always a concern. Please take the next 10 minutes and discuss with your table groups the suggestions here. Which do you already use? Which would you try? What additional strategies have you used that have proven effective? In 10 minutes we’ll come back as a large group and share some ideas.”

25. Tying It All Together (Making It Real) 1 of 2
Step One: Individual Work - Select one of the observations that you brought with you today. (It can be one from previous activities or a totally different one.) With your “Look Fors” from this morning, revise written feedback for Standard 3 to provide more specific feedback. (Write revised feedback on a separate piece of paper.)
Step Two: Exchange what you have written with someone at your table.
Step Three: Provide written feedback on your partner’s revised feedback. Does the written feedback
Avoid terms that express judgment?
Avoid words that imply quantity?
Stick to the five senses?
Indicate Who, What, When, Where, How? (Remember you do not have to describe everything, but the more descriptive you are, the more teachers will get from the feedback and the more evidence you will have for the summative rating.)
“We’re going to take the next bit of time to focus on tying our work together. We’ve done a lot of work on creating specific, targeted Look- Fors to be used as a basis for your teacher and principal observations as we move forward. The use of this tool is designed to improve your feedback to teachers in each of the standard areas. One important step to take is to personally assess your feedback from the past. This reflection will help you grow as we proceed.
Please go ahead and take out the samples of observations that you brought along today. You can choose to use documents you’ve already used with the Look For activities or to review something totally new. If you do not have an observation document, choose one or more of the performance indicators for Standard 3 and create written feedback based on an imaginary observation. Our first round will be individual. Select one observation and, with your newly formed Look- Fors in mind, revise your written feedback for Standard 3. Your revision should reflect your new understanding of Look Fors. You can write this revised statement on a separate sheet of paper. This will make it easier to share later.”
(Circulate and OBSERVE only. Provide 15 minutes to start and adjust time as needed.)
“OK, now that you have your revision, go ahead and exchange your statement with a partner at the table. Each of you will now provide your partner with written feedback on his/her revised statement. Specifically, let the person know if you feel the feedback: avoids terms that express judgment; avoids words that imply quantity; sticks to the five senses; and indicates who/what/when/where/how.”
(Circulate and OBSERVE only. Provide 10 minutes to start and adjust time as needed.)

26. Tying It All Together (Making It Real) 2 of 2
Step Four: Return observation feedback to original owner with your written feedback. Step Five: Incorporate your partner’s feedback into a final product on a clean sheet of paper. When finished, place the final document in the center of your table with your name on it. Facilitators will collect and redistribute to other tables. Step 6: Group Presentations – Each participant will share his/her assigned revised observation feedback with their table. Tables will provide feedback based upon bullets in Step Three. The participant will document the table’s feedback on the document to be returned to the original owner. Next Steps: Personal reflection: What are my strengths and challenges based upon the work from today?
“All right. Now go ahead and return your feedback to your partner.”
(Once all feedback is returned)
“Now let’s move on to step 5. This is time for another individual reflection. Go ahead and write yet another rendition of your feedback statement to the teacher that incorporates the feedback from your partner. Please do this on a new sheet of paper to avoid confusion. Place your final document in the middle of the table to be collected. You will put your name on the document so that it can be returned to you later.”
(Collect all products and redistribute to mixed tables).
“For step 6, each person at your table will share his/her randomly assigned piece of feedback. The group will then, in turn, provide additional feedback based on our criteria from step 3. Specifically, let the person know if you feel the feedback: avoids terms that express judgment; avoids words that imply quantity; stick to the five senses; and indicate who/what/when/where/how. While you are sharing, it is your responsibility to note the feedback on the document so that it can be returned to its original owner.”
(Circulate and OBSERVE only. Provide 20 minutes to start and adjust time as necessary.)
“Thank you for your work in supporting one another’s growth in providing feedback to teachers. At this point, please return the feedback document to the original owner. Now we will take a few minutes to quietly, individually reflect. This is the time, while your feedback is fresh, to assess your own strengths and challenges based on this work. This assessment will be the basis for your approach to our ongoing learning.”

27. Questions to Consider:
How Do We Know We Are Making Progress?
Questions to Consider:
Are we focusing on teaching and learning in terms of measurable student progress?
Have we developed a process for progress monitoring at the classroom level, the school level, and the division level?
Do we make adjustments based upon progress monitoring?
“As we will each time, we come back to these key questions for your consideration. How will you use these questions with your staff?”
(Solicit several responses before moving on.)

28. Classroom, School, Division
How Do We Know We Are Making Progress?
Classroom, School, Division
Set clear and measurable goals.
Implement action steps with fidelity.
Monitor and provide feedback.
Adjust based upon monitoring and data.
Repeat the cycle.
“How will you use this essential process in your building?”

29. Next Steps To be completed by Session 31.
“As agreed upon at the start, each time we meet, we will identify some next steps and give you an opportunity to identify next steps. We will model the importance of being specific - Who does what by when.”
(At this time the discussion should be led by the key person in the room who can approve and support the monitoring of next steps. Remind participants that they have next steps in their packet of slides so that they can take notes and record additional next steps, if any. Once next steps are identified, wrap up the day by reinforcing the first observation date. Also include any reminders that might be necessary to prepare for Session 3.)