Presentation on theme: "Measuring Educator Effectiveness for Music Teachers A Pennsylvania Perspective Created by O David Deitz for 2012-2013 PMEA Mentoring Program Webinar Series."— Presentation transcript:
Measuring Educator Effectiveness for Music Teachers A Pennsylvania Perspective Created by O David Deitz for PMEA Mentoring Program Webinar Series
(there are still a lot of questions to be asked and answered!) ?
House Bill 1901 (B) FOR PROFESSIONAL EMPLOYES AND TEMPORARY PROFESSIONAL EMPLOYES WHO SERVE AS CLASSROOM TEACHERS, THE FOLLOWING SHALL APPLY: (1) BEGINNING IN THE SCHOOL YEAR, THE EVALUATIO N OF THE EFFECTIVENESS OF PROFESSIONAL EMPLOYES AND TEMPORARY PROFESSIONAL EMPLOYES SERVING AS CLASSROOM TEACHERS SHALL GIVE DUE CONSIDERATION TO THE FOLLOWING: (I) CLASSROOM OBSERVATION AND PRACTICE MODELS THAT ARE RELATED TO STUDENT ACHIEVEMENT IN EACH OF THE FOLLOWING AREAS: (A) PLANNING AND PREPARATION. (B) CLASSROOM ENVIRONMENT. (C) INSTRUCTION. (D) PROFESSIONAL RESPONSIBILITIES. (II) STUDENT PERFORMANCE, WHICH SHALL COMPRISE FIFTY PER CENTUM (50%) OF THE OVERALL RATING OF THE PROFESSIONAL EMPLOYE OR TEMPORARY PROFESSIONAL EMPLOYE SERVING AS A CLASSROOM TEACHER AND SHALL BE BASED UPON MULTIPLE MEASURES OF STUDENT ACHIEVEMENT. Race to the Top
Charlotte Danielson’s Framework for Teaching Four Domains 22 Components 50% Observation/Evidence Danielson Framework Domains 1. Planning and Preparation 2. Classroom Environment 3. Instruction 4. Professional Responsibilities
8 Domain Focus— Adapted from Danielson’s Framework for Teaching 8 Planning and Preparation Classroom Environment Instruction Professional Responsibilities What a teacher knows and does in preparation for teaching. All aspects of teaching that lead to a culture for learning in the classroom. Professional responsibilities and behavior in and out of the classroom. What a teacher does to engage students in learning.
Multiple Measures of Teacher Effectiveness – Observation/Evidence and the Danielson Framework Classroom observations by Principal/supervisor, including evidence that demonstrates behaviors associated with improving student achievement: – Planning and preparation, including selecting standards-based lesson goals and designing effective instruction and assessment; – Classroom environment, including establishing a culture for learning and appropriate classroom management techniques that maximize instructional time; – Instruction, including the use of research-based strategies which engage students in meaningful learning and utilize assessment results to make decisions abut student needs; and – Professional responsibilities, including using systems for managing student data and communicating with student families 9
10 The Framework for Teaching The Framework for Teaching Charlotte Danielson Domain 1: Planning and Preparation a. Demonstrating Knowledge of Content and Pedagogy b. Demonstrating Knowledge of Students c. Setting Instructional Outcomes d. Demonstrating Knowledge of Resources e. Designing Coherent Instruction f. Designing Student Assessments Domain 2: The Classroom Environment a. Creating an Environment of Respect and Rapport b. Establishing a Culture for Learning c. Managing Classroom Procedures d. Managing Student Behavior e. Organizing Physical Space Domain 4: Professional Responsibilities a. Reflecting on Teaching b. Maintaining Accurate Records c. Communicating with Families d. Participating in a Professional Community e. Growing and Developing Professionally f. Showing Professionalism Domain 3: Instruction a. Communicating With Students b. Using Questioning and Discussion Techniques c. Engaging Students in Learning d. Using Assessment in Instruction e. Demonstrating Flexibility and Responsiveness
Danielson Framework, Domain 3: Instruction Component1. Failing2. Needs Improvement3. Proficient4. Distinguished 3a: Communicating with students Expectations for learning, directions and procedures, and explanations of content are unclear or confusing to students. Teacher’s use of language contains errors or is inappropriate to students’ cultures or levels of development Expectations for learning, directions and procedures, and explanations of content are clarified after initial confusion; teacher’s use of language is correct but may not be completely appropriate to students’ cultures or levels of development. Expectations for learning, directions and procedures, and explanations of content are clear to students. Communications are appropriate to students’ cultures and levels of development. Expectations for learning, directions and procedures, and explanations of content are clear to students. Teacher’s oral and written communication is clear and expressive, appropriate to students’ cultures and levels of development, and anticipates possible student misconceptions. 3b: Using questioning and discussion techniques Teacher’s questions are low-level or inappropriate, eliciting limited student participation, and recitation rather than discussion. Some of the teacher’s questions elicit a thoughtful response, but most are low-level, posed in rapid succession. Teacher’s attempts to engage all students in the discussions are only partially successful. Most of the teacher’s questions elicit a thoughtful response, and the teacher allows sufficient time for students to answer. The students are engaged and participate in the discussion, with the teacher stepping aside when appropriate. Questions reflect high expectations and are culturally and developmentally appropriate. Students formulate many of the high- level questions and ensure that all voices are heard. 11
Building Level Data PSSA Achievement PVAAS Growth Graduation Rate Promotion Rate Attendance Rate AP Course Participation SAT/PSAT Building Level Data PSSA Achievement PVAAS Growth Graduation Rate Promotion Rate Attendance Rate AP Course Participation SAT/PSAT 15 %
Teacher Specific/Elective Data Teacher Specific Data PSSA Achievement PVAAS Growth IEP Growth Locally Developed District Rubrics Elective Data/SLOs District Designed Measures and Examinations Nationally Recognized Standardized Tests Industry Certification Examinations Student Projects Pursuant to Local Requirements Student Portfolios Pursuant to Local Requirements 15% 20% 35% For MUSIC: Elective
What are some ways to develop evidence of student achievement in non-tested grades and subjects?
Student Learning ObjectivesOther AssessmentsMeasures of Collective Performance Three Approaches to the Challenge
Measures of Collective Performance School-wide growth measures (like the PA Report Card) Standardized assessments used to measure the performance of groups of teachers (like program evaluation)
Other Assessments Standardized end of course assessments, formative-benchmark-unit assessments Teacher-developed assessments of student learning or growth
Student Learning Objectives Content- and grade/course-specific measurable learning objectives that can be used to document student learning over a defined period of time Can be written to measure growth or mastery
NY, IN, MA, GA, UT, RI, NH, New Haven CT, Austin TX Student Learning Objectives FL, DE, Pittsburgh PA Other Assessments KY, Pittsburgh PA Measures of Collective Performance What are other states/schools using for non-tested grades and subjects?
What is PA planning to do ? Student Learning Objectives with Model Assessments
Why this approach? The “other assessment” approach causes narrowing of curriculum and is not financially sustainable. The “measures of collective performance” are already included in the formula (required by Race to the Top) and do not offer NTGS teachers an opportunity to be evaluated on what they actually teach. SLOs that don’t describe the assessment protocol tend to lack a true learning focus.
SLO Concepts Student achievement can be measured in ways that reflect authentic learning of content standards. Educator effectiveness can be measured through use of student achievement measures.
Educator Effectiveness: SLO >www.education.state.pa.us Measurement of Student Achievement Measurement of Educator Effectiveness
1.Know and understand the Standards PA Standards Common Core Standards Professional/Technical Standards 2. Assessment Tasks Authentic to the grade or course Aligned to Standards 3. Assessment Scoring Can describe levels of student achievement toward standards based learning objectives
THE PA SLO TEMPLATE & PROCESS What it is supposed to be:What it is not supposed to be: More paperwork for teachers that has no meaning or purpose More testing for students A weak substitute for PVAAS or other standardized testing data More paperwork for teachers that has no meaning or purpose More testing for students A weak substitute for PVAAS or other standardized testing data A format to inform strong instructional practice and strong student achievement A way to measure teacher effectiveness based on student achievement An opportunity for teachers to define, describe and present data on student achievement in the content area that they teach
PA Educator Challenge To develop and implement an appropriately rigorous measure of teacher effectiveness based on student achievement in your content area through the use of the PA SLO Template. >www.education.state.pa.us Educator Effectiveness: SLO
The Approach Provide a method for teachers to write SLOs. Provide a method for teachers to design assessments appropriate to the SLO. Provide piloted and peer-reviewed models on the PDE SAS website. Provide professional development appropriate to SLO development. The Methodology: AN SLO TEMPLATE
STUDENT LEARNING OBJECTIVE TEMPLATE A format to provide a measure of teacher effectiveness based on student achievement of content standards, as applicable to the “Elective Data” portion of the Teacher Effectiveness System in Act 82 (HB 1901). Educator Effectiveness: SLO >www.education.state.pa.us
Looking at the Components of PA’s SLO Template Teacher Information Content Area Student Learning Objective Data and Targets Used to Establish the SLO Assessment/Performance Task Administration of the Assessment/Performance Task Evidence of Individual Student Achievement Strategies/Actions to Achieve the SLO Implementation Timeline 30 Teacher Effectiveness Measure (Rating)
…one piece at a time. 1.Teacher Information Teacher Name School Name District Name 2. Content Area Course Title or Content Area PA Standards Subject Area Grade Level(s) Brief Description of the course/ content area Number of students per class/session Frequency of classes/sessions Number of minutes per class/session Total number of classes/sessions
9. Teacher Effectiveness Measure Classroom Objective How will the aggregated scores of the “Evidence of Individual Student Achievement” results be used to define teacher effectiveness? Failing: few students achieve content mastery or growth Needs Improvement: less than a significant number of students achieve content mastery or growth Proficient: A significant number of students achieve content mastery or growth Distinguished: An exceptional number of students achieve content mastery or growth
Grade 8 General Music Less than 60% of students will score on a minimum of 2 assessments, with scores lower than 37 on the other two assessments. 60%-70% of students will score on a minimum of 2 assessments, with scores lower than 37 on the other two assessments. 70%-84% of students will score on a minimum on 2 out of the four assessments, and score no lower than 37 on the other two assessments. 85% of students will score on a minimum of 3 out of the four assessments, and score no lower than 37 on a fourth assessment. HS Instrumental Music Mastery: Less than 50% of students achieve proficient or advanced levels in 5 out of 6 components on the second assessment. Growth: Less than 50% of students achieve improved scores in at least 2 out of 6 components, with no decrease in scores in the other components. Mastery: 50% -74% of students achieve proficient or advanced levels in 5 out of 6 components. Growth: 50% -74% of students achieve improved scores in at least 2 out of 6 components, with no decrease in scores in the other components.. Mastery: 70% -84% of students achieve proficient or advanced levels in 5 out of 6 components Growth: 70% -84% of students achieve improved scores in 2 out of 6 components, with no decrease in scores in the other components Mastery: 85% -100% of students achieve proficient or advanced levels in 5 out of 6 components Growth: 85% -100% of students achieve improved scores in 2 out of 6 components, with no decrease in the scores in other components
Student Learning Objective Guiding Questions: Why is this objective important and meaningful to your students? Sample Statements: Grade 8 General Music: Students will utilize musical notation as both a performer and composer. HS Instrumental Music: Students develop independent performance skills appropriate to positive musical contribution in an instrumental performing ensemble. 3.SLO (Student Learning Objective) Learning objectives that can be validly measured to document student learning over a defined period of time.
SLO Rationale Guiding Questions: How is your student learning objective measurable and clearly understood by students in this grade/course? Based upon this objective, how will students independently demonstrate their learning? Sample Statements: Grade 8 General Music: Students will independently demonstrate their learning by performing individually on the piano and guitar and by creating an individual musical composition based on the structural designs found in the music of the classic period ( ). HS Instrumental Music: The quality and musicality of an instrumental ensemble is dependent on each individual as a contributor to the collective whole. Individual assessments inform each student of his/her progress and contribution as an independent musician. 3.SLO (Student Learning Objective) Learning objectives that can be validly measured to document student learning over a defined period of time.
Curriculum Framework Connection What are the big ideas, essential questions, concepts and competencies (as identified by PA’s Curriculum Framework) to be measured? Essential Questions Concepts Competencies Big Ideas Professional/Industry Standards Common Core Standards PA Standards Statements from this section should be cut and pasted from :
Student Preparedness/ Baseline Data Guiding Questions: What, if any, are the course or grade prerequisites? What do you know about the students in this class at the beginning of the learning process? What may typical and not-so-typical student progress look like on the way to achieving the learning objective? Sample Statements: Grade 8 General Music: There are no specific prerequisites. They will have had some instruction in music history, composition, guitar, and keyboard in grades six and seven. Large lapses of time between instruction periods (up to a year) will cause a need for extensive review of concepts and competencies previously experienced. Most students will be successful at manipulating the instruments and writing music notation. Some students will struggle with conceptualizing ideas regarding music composition. A pretest of retained knowledge of music notation is administered on the first day of the course to inform instruction. 4. Data and Targets Used to Establish the SLO
Targets Guiding Questions: What is the expected outcome or goal by the end of the designated instructional period (i.e., year, semester, course length, etc.)? Using student baseline data, are differentiated targets/goals needed? If yes, what are those targets/goals? Are the targets ambitious yet realistic, and how will you monitor progress along the way? Sample Statements: HS Instrumental Music: Each student will demonstrate an ability to perform his/her instrumental part independently in an unaccompanied setting, exhibiting mastery and/or growth in such components as tone quality, intonation, rhythm, musicality, and technique appropriate to the student’s specific instrument.
5. Assessment/Performance Task Name of the Assessment/ Performance Task Sample Statements: Grade 8 General Music: Classical Time Period Assessment Composition Assessment Keyboard Performance Assessment Guitar Performance Assessment HS Instrumental Music: HS Instrumental Music-Individual Student Performance
5. Assessment/Performance Task Description of the Assessment/ Performance Task Guiding Questions: Who is the developer of the assessments/performance task used (e.g., teacher-made, district-developed, commercial, etc.)? What is a description of the assessment/performance task that will be used to measure the student learning objective (SLO)? Are there any products or artifacts that will be gathered as part of the data collection process? Sample Statements: HS Instrumental Music: Individual Performance Assessments (IPA): The IPA will occur two times during the year, near the mid-point of the first and fourth marking periods. Students will usually submit the assessment electronically, a variety of submission formats will be available.. Material for this assessment will be drawn from both scales and ensemble music currently being rehearsed in class. Specific technical exercises taught and rehearsed in class may be included as well. IPA Requirements, First Marking Period 2012 Wind Symphony
Growth or Mastery Does this assessment/ performance task align with the student achievement goals for the SLO? Check one: Growth (change in student achievement across two or more points in time) Mastery (attainment of a defined level of achievement) Growth and Mastery >www.education.state.pa.us Educator Effectiveness: SLO
6. Administration of the Assessment/Performance Task Frequency of Assessment/ Performance Task Administration Guiding Questions: How often and when is this assessment/performance task administered? If measuring growth, are multiple assessment windows in place? Sample Statements: Grade 8 General Music: Each of the four project assessments is administered once throughout the course. Formative assessments are inherent in the instructional design. A pretest is administered on the first day of class that assesses prior knowledge applicable to the composition and keyboard assessments. HS Instrumental Music: Students will be assessed at the midpoint of the first and fourth quarters.
6. Administration of the Assessment/Performance Task Resources Required Guiding Questions: What unique or specific equipment, technologies, or resources are needed to complete this assessment/performance task? Sample Statements: Grade 8 General Music: Keyboards, guitars, audio-playback equipment. HS Instrumental Music: Video/audio recording technology and/or methodologies for students to create online submissions that will require computers and appropriate software.
All Classes: IEP and 504 accommodations will be implemented. 6. Administration of the Assessment/Performance Task Adaptations for Diverse Learners and/or Students with Disabilities Guiding Questions: What assessment/performance task adaptations are needed to assist diverse learners and/or students with disabilities? Sample Statements: Grade 8 General Music: Students struggling with keyboard or guitar playing skills can perform simple chord structures. Students can use a computer notation program to create the composition. HS Instrumental Music: IEP adaptations are made (additional time allotted, alternative print formats, teacher-directed instruction to help students feel more comfortable about being assessed individually, etc.)
Most Classes: The assessments can be administered by an equivalent peer. Some Classes: Student use of equipment needed to complete the authentic assessment could be monitored by an equivalent peer, but should probably be monitored by the class instructor (i.e. athletic, career tech, chemistry, biology equipment). 6. Administration of the Assessment/Performance Task Personnel Guiding Questions: Can this assessment/performance task be administered by an equivalent peer (educator in a similar content area)? If not, please explain. Does a district policy exist with regard to assessment/performance task administration? Sample Statements: An equivalent peer can administer this assessment. Many students will self- administer this assessment.
7. Evidence of Student Achievement Rubrics/ Scoring Scales Guiding Questions: How will individual student growth or mastery be determined (defined and scored) using this assessment/performance task? Include the specific rubric/scoring scale that will be used. Does the rubric and/or scoring scale correlate with the assessment/performance task? Sample Statements: HS Instrumental Music: See next slide…………. OR LOOK AT SCOTT SHEEHAN’S PMEA WEBINAR PRESENTATION
7. Evidence of Student Achievement CategoryAdvancedProficientBasicBelow Basic Tone Quality Tone quality is controlled, full, rich, and characteristic in all registers Tone quality is characteristic in the normal playing range of the instrument, but distorts in a few passages or extreme registers (occasional lapses of control), these lapses rarely hinder the performance. Tone quality exhibits some flaws in production in the normal range of the instrument, extreme registers are usually uncontrolled, tone quality occasionally detracts from performance. Tone quality has significant flaws in basic production in all registers of the instrument and consistently detracts from the performance. IntonationIntonation is accurate in all ranges and registers Intonation is mostly accurate; the few problem pitches are adjusted to an acceptable standard during the performance. Intonation is somewhat accurate but consistently includes out-of-tune notes; these inaccuracies occasionally detract from the performance. Intonation is consistently inaccurate and hinders the quality of performance. Rhythmic Accuracy Rhythms are accurate and precise throughout the performance. Rhythms are nearly accurate; occasionally, rhythms lack precise interpretation. Most rhythm patterns are accurate, but errors in precision are present and occasionally detract from the performance. Many rhythms performed incorrectly or inconsistently, major errors are present and detract from the performance. Pitch Accuracy (melodic accuracy) Pitches are consistently correct. Occasional incorrect pitches are played, but do not detract from the performance Most pitches are correct, but errors are present and occasionally detract from the performance. Many pitches are performed incorrectly; this inaccuracy consistently seriously hinders the performance.
7. Evidence of Student Achievement Data Collection Guiding Questions: In what format will data be collected (e.g., database, graphed, portfolio, etc.) Is a pre-post test being used? (If so, please describe.) How frequently will data be collected? Scores from the rubric will be collected on a database. Data will be collected at the mid-point of the first and fourth marking periods.
7. Evidence of Student Achievement Scoring Student Progress/ Preparation Guiding Questions: How was baseline data collected? (If baseline data was not collected, please explain.) Can baseline data be compared with the results of this assessment/performance task? Sample Statements: Grade 8 General Music: Baseline data is collected through a pretest of retained knowledge; there is no assessment of authentic skills prior to instruction. HS Instrumental Music: Scoring from the first marking period assessment will be used as baseline data for describing growth. Scoring from the fourth marking period assessment will be used to describe growth and/or mastery.
7. Evidence of Student Achievement Data Presentation Guiding Questions: What evidence will be presented to principal/evaluator to support the teacher effectiveness measure? How will data be presented to the principal/evaluator (e.g., database, graphed, portfolio, individual student artifacts, etc.)? Sample Statements: Grade 8 General Music: Evidence will be presented through aggregated scores from the database. HS Instrumental Music: An ipad database program created by the teacher will be used to present scoring information.
All Grades and Courses: The assessment can be scored by an equivalent peer. All Grades and Courses: The assessment is a commercially produced assessment that has a history of validity and reliability. Data Analysis and Interpretation Guiding Questions: How can the assessment/performance task results be interpreted in the same way across equivalent peers? Is there a reliable and valid scoring and interpretive process (i.e., state developed, district-based, commercial, standardized, etc.) that is associated with the assessment/performance task? If so, please describe. Sample Statements: An equivalent peer can use the rubrics to assess the projects. 7. Evidence of Student Achievement
8. Strategies/Actions to Achieve the SLO Assessment for Learning Guiding Questions: What formative assessment information lets you know if your instructional practices will lead to successful completion of the SLO? Sample Statements: Grade 8 General Music: Rubric will be provided to the students as a part of progress monitoring throughout the instructional period. HS Instrumental Music: Interim formative assessments during group lessons will be used to focus instruction toward successful completion of the SLO.
8. Strategies/Actions to Achieve the SLO Alignment with the Danielson Framework for Teaching Guiding Questions: Based upon reflection, what instructional practices would you like to change or strengthen? What professional learning and/or other type of support will help you to achieve this SLO? Sample Statements: Grade 8 General Music: Based on the formative assessment information provided by student composition and performance tasks, students will be provided with supplemental materials to either address music reading difficulties or enhance current abilities to go beyond the targets. (Danielson 3E) HS Instrumental Music: Based on weekly formative performance assessments, student seating within the band will be uniquely rearranged periodically to provide students the opportunity to develop higher levels of performance independence. (Danielson 2 E)