Presentation on theme: "Measuring Educator Effectiveness for Music Teachers"— Presentation transcript:
1 Measuring Educator Effectiveness for Music Teachers A Pennsylvania PerspectiveCreated by O David Deitz for PMEA Mentoring Program Webinar Series
2 (there are still a lot of questions to be asked and answered!) ?(there are still a lot of questions to be asked and answered!)
3 An effective teacher in every classroom. Big Idea:An effective teacher in every classroom.Measuring Teacher Effectiveness Based on Student Achievement of Content Standards is one of several processes being used in Pennsylvania to reach a very simply stated yet specific Big Idea: every student deserves to have an effective teacher guiding his or her instructional pursuits. Research over past years has shown differences in student academic growth based on the level of effectiveness of that student’s teacher.
4 House Bill 1901 Race to the Top STUDENT ACHIEVEMENT. (B) FOR PROFESSIONAL EMPLOYES AND TEMPORARY PROFESSIONALEMPLOYES WHO SERVE AS CLASSROOM TEACHERS, THE FOLLOWING SHALL APPLY:(1) BEGINNING IN THE SCHOOL YEAR, THE EVALUATIO NOF THE EFFECTIVENESS OF PROFESSIONAL EMPLOYES AND TEMPORARYPROFESSIONAL EMPLOYES SERVING AS CLASSROOM TEACHERS SHALL GIVEDUE CONSIDERATION TO THE FOLLOWING:(I) CLASSROOM OBSERVATION AND PRACTICE MODELS THAT ARERELATED TO STUDENT ACHIEVEMENT IN EACH OF THE FOLLOWING AREAS:(A) PLANNING AND PREPARATION.(B) CLASSROOM ENVIRONMENT.(C) INSTRUCTION.(D) PROFESSIONAL RESPONSIBILITIES.(II) STUDENT PERFORMANCE, WHICH SHALL COMPRISE FIFTY PERCENTUM (50%) OF THE OVERALL RATING OF THE PROFESSIONAL EMPLOYEOR TEMPORARY PROFESSIONAL EMPLOYE SERVING AS A CLASSROOM TEACHERAND SHALL BE BASED UPON MULTIPLE MEASURES OFSTUDENT ACHIEVEMENT.Race to the TopHouse Bill 1901
5 States across the country are in the process of developing and implementing teacher evaluation systems that look at teacher effectiveness as a complex set of skills. This requires a variety of measures to describe that effectiveness. The multi-measure system in Pennsylvania looks at several different types of data, including principal observation of teaching behaviors, teacher contributions to the academic life of the building in which he or she teaches, specific data found in the state standardized testing, and data based on student achievement in ways not assessed through standardized testing. The student learning objective process is closely tied to the “elective data” portion of our state’s teacher evaluation system. This process shows educators how student achievement on district-developed common assessments, nationally recognized standardized tests, industry certification exams, student projects and student portfolios can be used to measure teacher effectiveness. Teachers of state standardized tested grades and subjects will have 20% attribution of Elective Data to their overall teacher evaluation.
6 Teachers of non-tested grades and subjects--possibly as high as 70% of the teaching staff in a school or district—will attribute 35% of the elective data to their overall teacher evaluation.
7 Charlotte Danielson’s Framework for Teaching Observation/EvidenceDanielson Framework DomainsPlanning and PreparationClassroom EnvironmentInstructionProfessional ResponsibilitiesFour Domains 22 Components 50%
8 Domain Focus— Adapted from Danielson’s Framework for Teaching Planning and PreparationProfessionalResponsibilitiesClassroomEnvironmentInstructionProfessional responsibilities and behavior in and out of the classroom.What a teacher knows and does in preparation for teaching.Here they are – how did you do with your recall?All aspects of teaching that lead to a culture for learning in the classroom.What a teacher does to engage students in learning.88
9 Multiple Measures of Teacher Effectiveness – Observation/Evidence and the Danielson Framework Classroom observations by Principal/supervisor, including evidence that demonstrates behaviors associated with improving student achievement:Planning and preparation, including selecting standards-based lesson goals and designing effective instruction and assessment;Classroom environment, including establishing a culture for learning and appropriate classroom management techniques that maximize instructional time;Instruction, including the use of research-based strategies which engage students in meaningful learning and utilize assessment results to make decisions abut student needs; andProfessional responsibilities, including using systems for managing student data and communicating with student families
10 The Framework for Teaching Domain 1: Planning and Preparationa. Demonstrating Knowledge of Content andPedagogyb. Demonstrating Knowledge of Studentsc. Setting Instructional Outcomesd. Demonstrating Knowledge of Resourcese. Designing Coherent Instructionf. Designing Student AssessmentsDomain 2: The Classroom Environmenta. Creating an Environment of Respect andRapportb. Establishing a Culture for Learningc. Managing Classroom Proceduresd. Managing Student Behaviore. Organizing Physical SpaceDomain 4: Professional Responsibilitiesa. Reflecting on Teachingb. Maintaining Accurate Recordsc. Communicating with Familiesd. Participating in a Professional Communitye. Growing and Developing Professionallyf. Showing ProfessionalismDomain 3: Instructiona. Communicating With Studentsb. Using Questioning and DiscussionTechniquesc. Engaging Students in Learningd. Using Assessment in Instructione. Demonstrating Flexibility andResponsivenessThis is the framework again that will constitute a significant percentage of your evaluation with the components that need to be further operationalized given your unique role and function….so, take a moment and look at Domain 1/Planning and Preparation –1. what would you be doing in your role and function for this domain in general?2. I demonstrate knowledge of content and pedagogy (1a.) by ____________ and here’s what I would be doing in order to be evaluated as “Proficient”……**we need a handout with these questionsThe Framework for Teaching Charlotte Danielson10
11 Danielson Framework, Domain 3: Instruction Component1. Failing2. Needs Improvement3. Proficient4. Distinguished3a: Communicating with studentsExpectations for learning, directions and procedures, and explanations of content are unclear or confusing to students. Teacher’s use of language contains errors or is inappropriate to students’ cultures or levels of developmentExpectations for learning, directions and procedures, and explanations of content are clarified after initial confusion; teacher’s use of language is correct but may not be completely appropriate to students’ cultures or levels of development.Expectations for learning, directions and procedures, and explanations of content are clear to students. Communications are appropriate to students’ cultures and levels of development.Expectations for learning, directions and procedures, and explanations of content are clear to students. Teacher’s oral and written communication is clear and expressive, appropriate to students’ cultures and levels of development, and anticipates possible student misconceptions.3b: Using questioning and discussion techniquesTeacher’s questions are low-level or inappropriate, eliciting limited student participation, and recitation rather than discussion.Some of the teacher’s questions elicit a thoughtful response, but most are low-level, posed in rapid succession. Teacher’s attempts to engage all students in the discussions are only partially successful.Most of the teacher’s questions elicit a thoughtful response, and the teacher allows sufficient time for students to answer. The students are engaged and participate in the discussion, with the teacher stepping aside when appropriate.Questions reflect high expectations and are culturally and developmentally appropriate. Students formulate many of the high-level questions and ensure that all voices are heard.
12 15 % Building Level Data Building Level Data PSSA Achievement PVAAS GrowthGraduation RatePromotion RateAttendance RateAP Course ParticipationSAT/PSAT15 %Building Level DataPSSA AchievementPVAAS GrowthGraduation RatePromotion RateAttendance RateAP Course ParticipationSAT/PSAT
13 Teacher Specific/Elective Data Teacher Specific DataPSSA AchievementPVAAS GrowthIEP GrowthLocally Developed District Rubrics15%Elective Data/SLOsDistrict Designed Measures and ExaminationsNationally Recognized Standardized TestsIndustry Certification ExaminationsStudent Projects Pursuant to Local RequirementsStudent Portfolios Pursuant to Local Requirements20%For MUSIC: Elective35%
14 What are some ways to develop evidence of student achievement in non-tested grades and subjects?
15 Three Approaches to the Challenge Measures of Collective PerformanceOther AssessmentsStudent Learning Objectives
16 Measures of Collective Performance School-wide growth measures (like the PA Report Card)Standardized assessments used to measure the performance of groups of teachers (like program evaluation)
17 Teacher-developed assessments of student learning or growth Other AssessmentsStandardized end of course assessments, formative-benchmark-unit assessmentsTeacher-developed assessments of student learning or growth
18 Student Learning Objectives Content- and grade/course-specific measurable learning objectives that can be used to document student learning over a defined period of timeCan be written to measure growth or mastery
19 What are other states/schools using for non-tested grades and subjects? Student Learning ObjectivesNY, IN, MA, GA, UT, RI, NH, New Haven CT, Austin TXOther AssessmentsFL, DE, Pittsburgh PAMeasures of Collective PerformanceKY, Pittsburgh PA
20 What is PA planning to do? Student Learning Objectives with Model Assessments
21 Why this approach?The “other assessment” approach causes narrowing of curriculum and is not financially sustainable. The “measures of collective performance” are already included in the formula (required by Race to the Top) and do not offer NTGS teachers an opportunity to be evaluated on what they actually teach. SLOs that don’t describe the assessment protocol tend to lack a true learning focus.
22 SLO ConceptsStudent achievement can be measured in ways that reflect authentic learning of content standards.Educator effectiveness can be measured through use of student achievement measures.The concept that student achievement can be measured is not new. Standardized testing, grades, achievement rubrics and academic rewards systems are founded on the premise that student achievement can be measured. But new ideas about measurement of student achievement have surfaced as the goals of education focus on 21st century learning skills and college and career readiness: first, there are standards upon which authentic student learning in a given content area should be measured, and second, teacher effectiveness can be measured based upon student achievement measures.
23 Standards Based Education Educator Effectiveness: SLOStandards Based EducationMeasurement of Student AchievementMeasurement of Educator EffectivenessThe Student Learning Objective process creates a relationship between student achievement and teacher effectiveness under the umbrella of standards based education. An SLO is about linking student achievement of measurable, standards aligned objectives to effective teacher practice, and strong SLOs will inform effective teacher practice as well as provide measurement of student achievement and teacher effectiveness.>
24 What knowledge and skills might be needed to develop an SLO? While writing SLOs does not require a lot of specific or sophisticated educational jargon, it does require some knowledge about content standards and some skills in developing appropriate assessments to describe student achievement of content standards.
25 Know and understand the Standards PA Standards Common Core Standards Professional/Technical2. Assessment TasksAuthentic to the gradeor courseAligned to Standards3. Assessment ScoringCan describe levels of student achievement toward standards based learning objectivesKnowing and understanding the standards for the grade or course being taught is essential for writing SLOs. School districts who have worked to align their curriculum with PA Standards, Common Core Standards and/or Professional/Technical Standards in all content areas will have already conquered the first hurdle.A working familiarity with assessment tasks appropriate to the grade or course being taught is also essential. This statement is not meant to imply that MORE assessments should be created; teachers should be looking at assessments that are already in place and evaluating the authenticity that the assessment tasks have to the learning objectives in that grade or course, as well as the assessment’s alignment to standards.A skill closely related to the assessment task is the development of scoring methodologies that are well aligned to both the task and the learning objectives assessed by that task. Scoring scales and rubrics that clearly describe to what extent a student is achieving the learning objectives being assessed by the task will be important in describing to what extent the teacher is effective in instructing students toward achieving those objectives.
26 THE PA SLO TEMPLATE & PROCESS What it is supposed to be:What it is not supposed to be:A format to inform strong instructional practice and strong student achievementA way to measure teacher effectiveness based on student achievementAn opportunity for teachers to define, describe and present data on student achievement in the content area that they teachMore paperwork for teachers that has no meaning or purposeMore testing for studentsA weak substitute for PVAAS or other standardized testing dataMore paperwork for teachers that has no meaning or purposeMore testing for studentsA weak substitute for PVAAS or other standardized testing dataThe SLO Template and Process are designed to help teachers positively demonstrate the ways in which they are effective through valuable learning objectives that are content specific. Initial attempts to create SLOs will be both thought provoking and time consuming but should inform strong instructional practice, this being a primary goal of teacher effectiveness. A quality SLO can inform strong instructional practice without increasing the amount of testing that students and teachers currently experience.
27 Educator Effectiveness: SLO PA Educator ChallengeTo develop and implement an appropriately rigorous measure of teacher effectiveness based on student achievement in your content area through the use of the PA SLO Template.So, now is the time to begin a task that, once completed, will authentically demonstrate an ability to create an appropriately rigorous measure of teacher effectiveness based on student achievement for the grade or courses that you teach. The tool developed to guide teachers through this process is call the Student Learning Objectives, or SLO Template. Please remember that this tool, much like many parts of the teacher evaluation process, is being piloted during the school year and will continue to undergo changes as Pennsylvania and other states work with this process. It is hoped that principals will encourage teachers to complete this template and implement the process in specific content areas. It is also hoped that teachers are encouraged to share their models with colleagues and submit them to a Pennsylvania Dept. of Education resource for review, comment and future inclusion on a webpage of model SLOs.>
28 The Approach The Methodology: AN SLO TEMPLATE Provide a method for teachers to write SLOs.Provide a method for teachers to design assessments appropriate to the SLO.Provide piloted and peer-reviewed models on the PDE SAS website.Provide professional development appropriate to SLO development.The Methodology:AN SLO TEMPLATE
29 STUDENT LEARNING OBJECTIVE TEMPLATE Educator Effectiveness: SLOSTUDENT LEARNING OBJECTIVE TEMPLATEA format to provide a measure of teacher effectiveness based on student achievement of content standards, as applicable tothe “Elective Data” portion of the Teacher Effectiveness Systemin Act 82 (HB 1901).An opening description of the student learning objective template informs teachers that, while student learning is the primary goal of all instructional processes, this is also methodology to measure teacher effectiveness through evidence of student achievement. The student learning objective process, including a complete template, approval by the principal or teacher evaluator, and implementation and completion of the process, is applicable to the “elective portion” of the teacher evaluation system.>
30 Looking at the Components of PA’s SLO Template Teacher InformationContent AreaStudent Learning ObjectiveData and Targets Used to Establish the SLOAssessment/Performance TaskAdministration of the Assessment/Performance TaskEvidence of Individual Student AchievementStrategies/Actions to Achieve the SLOImplementation TimelineThe SLO Template, which you should have printed out or have available to you electronically as we go through the process, has twelve basic sections, each designed to help frame reflection about the students you are teaching, the setting in which they are being taught, the nature of the content students are experiencing, the goals and objectives you have for their learning, the ways in which you will know students are achieving appropriate objectives, strategies to improve instruction and a description of your effectiveness as a teacher based on your students’ demonstration of student achievement. Beginning with the end in mind, let’s jump to section 9, the Teacher Effectiveness Measure.Teacher Effectiveness Measure (Rating)
31 …one piece at a time. Teacher Information Teacher Name School Name School NameDistrict Name2. Content AreaCourse Title or Content AreaPA Standards Subject AreaGrade Level(s)Brief Description of the course/ content areaNumber of students per class/sessionFrequency of classes/sessionsNumber of minutes per class/sessionTotal number of classes/sessions
32 9. Teacher Effectiveness Measure Classroom ObjectiveHow will the aggregated scores of the “Evidence of Individual Student Achievement” results be used to define teacher effectiveness?Failing: few students achieve content mastery or growthNeeds Improvement: less than a significant number of students achieve content mastery or growthProficient: A significant number of students achieve content mastery or growthDistinguished: An exceptional number of students achieve content mastery or growthLet’s look at the Teacher Effectiveness Measure from the classroom objective perspective. Statements of collective student achievement would be developed for this section, and categorized in the same way that statements from the Danielson Framework for Teaching would be categorized: Distinguished, Proficient, Needs Improvement and Failing. The following slide will demonstrate how these statements would be written for both the Grade 1 Art and Grade 8 Visual Art.
33 Grade 8 General Music HS Instrumental Music Less than 60% of students will score on a minimum of 2 assessments, with scores lower than 37 on the other two assessments.60%-70% of students will score on a minimum of 2 assessments, with scores lower than 37 on the other two assessments.70%-84% of students will score on a minimum on 2 out of the four assessments, and score no lower than 37 on the other two assessments.85% of students will score on a minimum of 3 out of the four assessments, and score no lower than 37 on a fourth assessment.HS Instrumental MusicMastery: Less than 50% of students achieve proficient or advanced levels in 5 out of 6 components on the second assessment.Growth: Less than 50% of students achieve improved scores in at least 2 out of 6 components, with no decrease in scores in the other components. Mastery: 50% -74% of students achieve proficient or advanced levels in 5 out of 6 components.Growth: 50% -74% of students achieve improved scores in at least 2 out of 6 components, with no decrease in scores in the other components..Mastery: 70% -84% of students achieve proficient or advanced levels in 5 out of 6 componentsGrowth: 70% -84% of students achieve improved scores in 2 out of 6 components, with no decrease in scores in the other componentsMastery: 85% -100% of students achieve proficient or advanced levels in 5 out of 6 componentsGrowth: 85% -100% of students achieve improved scores in 2 out of 6 components, with no decrease in the scores in other componentsNotice that both types of teacher effectiveness statements—the Grade 1 Math and the Grade 8 Art--describe what assessments are used and how many students, in percentages, will achieve certain levels of proficiency on those assessments. Classroom objective statements could also describe change in student achievement over time, when appropriate. On the next slide you will see a targeted objective statement, designed to focus instruction and subsequent achievement for students in unique academic populations.
34 SLO (Student Learning Objective) Learning objectives that can be validly measured to document student learning over a defined period of time.StudentLearningObjectiveGuiding Questions:Why is this objective important and meaningful to your students?Sample Statements:Grade 8 General Music: Students will utilize musical notation as both a performer and composer.HS Instrumental Music: Students develop independent performance skills appropriate to positive musical contribution in an instrumental performing ensemble.Although the teacher effectiveness measure is an important product of the process, the student learning objective is the focal point. The objective should be both meaningful and measureable, reflecting a student’s ability to independently demonstrate learning of content-based standards. The objective can be written to address the entire scope of a grade or course, or could be written to address a specific need as demonstrated by prior knowledge of student learning. The student learning objective is the statement around which the rest of the process unfolds, so the strength and clarity of the objective is of great importance.
35 SLO (Student Learning Objective) Learning objectives that can be validly measured to document student learning over a defined period of time.SLO RationaleGuiding Questions:How is your student learning objective measurable and clearly understood by students in this grade/course?Based upon this objective, how will students independently demonstrate their learning?Sample Statements:Grade 8 General Music: Students will independently demonstrate their learning by performing individually on the piano and guitar and by creating an individual musical composition based on the structural designs found in the music of the classic period ( ).HS Instrumental Music: The quality and musicality of an instrumental ensemble is dependent on each individual as a contributor to the collective whole. Individual assessments inform each student of his/her progress and contribution as an independent musician.As promised, the process provides many opportunities to check for alignment, and the SLO rationale is the first one. This statement begins to preview the nature of the tasks and assessments that will be used to describe student achievement of the learning objective.
36 Statements from this section should be cut and pasted from: Curriculum Framework ConnectionWhat are the big ideas, essential questions, concepts and competencies (as identified by PA’s Curriculum Framework) to be measured?Big IdeasStatements from this section should be cut and pasted from:Essential QuestionsConceptsCompetenciesPA StandardsCommon Core StandardsProfessional/Industry Standards
37 4. Data and Targets Used to Establish the SLO Student Preparedness/Baseline DataGuiding Questions:What, if any, are the course or grade prerequisites?What do you know about the students in this class at the beginning of the learning process?What may typical and not-so-typical student progress look like on the way to achieving the learning objective?Sample Statements:Grade 8 General Music: There are no specific prerequisites. They will have had some instruction in music history, composition, guitar, and keyboard in grades six and seven. Large lapses of time between instruction periods (up to a year) will cause a need for extensive review of concepts and competencies previously experienced. Most students will be successful at manipulating the instruments and writing music notation. Some students will struggle with conceptualizing ideas regarding music composition. A pretest of retained knowledge of music notation is administered on the first day of the course to inform instruction.The Data and Targets section of the SLO template begins by looking at what a teacher knows about the students he or she is teaching. For some grades and courses, very specific content testing data or course prerequisites may exist. For other grades and courses, very little background about the specific students in the class may be known, but past experiences in teaching the course may inform anticipated targets for student achievement.
38 4. Data and Targets Used to Establish the SLO Guiding Questions:What is the expected outcome or goal by the end of the designated instructional period (i.e., year, semester, course length, etc.)?Using student baseline data, are differentiated targets/goals needed? If yes, what are those targets/goals?Are the targets ambitious yet realistic, and how will you monitor progress along the way?Sample Statements:HS Instrumental Music: Each student will demonstrate an ability to perform his/her instrumental part independently in an unaccompanied setting, exhibiting mastery and/or growth in such components as tone quality, intonation, rhythm, musicality, and technique appropriate to the student’s specific instrument.Based on knowledge of student preparedness and the amount of time the teacher has to deliver instruction, an SLO should have ambitious yet attainable goals, known as “targets,” that will help to define achievement for students in the class. Clear target statements will provide a focus to the student learning objective and will also help to frame the tasks and assessments developed to evaluate student achievement of the objective.
39 5. Assessment/Performance Task Name of the Assessment/Performance TaskSample Statements:Grade 8 General Music:Classical Time Period AssessmentComposition AssessmentKeyboard Performance AssessmentGuitar Performance AssessmentHS Instrumental Music:HS Instrumental Music-Individual Student PerformanceThank you for your participation in the activity. Please leave the yellow cards on the table, as your input will help us to provide clearer presentations in the future.The next segment of this presentation will focus on the assessment/performance task. Using the word “test” would be a lot shorter, but would not adequately describe some very specific types of authentic assessments, like projects and portfolios, that are inherent in many content areas. We’ll take a look at some task statements and their abilities to describe mastery of or growth toward achieving content-specific objectives. We’ll also look at some ideas regarding task administration and the importance of ongoing formative assessments as both checks for understanding and guidance toward improving instruction.The current SLO template asks for the assessment/performance task to have a name. Many teacher-developed tasks may not have a name, and a name may not be necessary for the teacher and the principal, but should a teacher submit and SLO as a model for future inclusion on the state website, we would need a name so that the task could be appropriately linked to the SLO.
40 5. Assessment/Performance Task Descriptionof theAssessment/PerformanceTaskGuiding Questions:Who is the developer of the assessments/performance task used (e.g., teacher-made, district-developed, commercial, etc.)?What is a description of the assessment/performance task that will be used to measure the student learning objective (SLO)?Are there any products or artifacts that will be gathered as part of the data collection process?Sample Statements:HS Instrumental Music: Individual Performance Assessments (IPA):The IPA will occur two times during the year, near the mid-point of the first and fourth marking periods. Students will usually submit the assessment electronically, a variety of submission formats will be available.. Material for this assessment will be drawn from both scales and ensemble music currently being rehearsed in class. Specific technical exercises taught and rehearsed in class may be included as well.IPA Requirements, First Marking Period 2012Wind SymphonyAs you can see with the task description statement, commercially developed assessments tend to be easier to describe than do teacher developed assessments! Tasks for authentic assessments may initially require a lot of explanation, until principals become familiar with the assessment protocols that their teachers are using. As an interesting aside, the bottom line of the Grade 8 Art Task is a link to the PDESAS.org website, where this teacher has provided a complete lesson plan for the project in the Publish Your Best component of the Materials and Resources Section. At the end of today’s presentation you will see a short list of SLO and authentic task and assessment resources that, like the PDESAS.org resource, are helpful toward assisting teachers to write tasks, assessments, SLOs and standards-based instructional units.
41 Educator Effectiveness: SLO Growth or MasteryDoes this assessment/performance task align with the student achievement goals for the SLO?Check one:Growth (change in student achievementacross two or more points in time)Mastery (attainment of a defined level ofachievement)Growth and MasteryThe “Growth or Mastery” statements can help teachers to create accurate statements about the type of achievement that should be expected from the learning objective. If the goal is growth, then student achievement levels at the beginning of the learning process must be well defined. If mastery is the goal, then the target must be clearly stated. Teachers will find that for some student populations, descriptions of both mastery and growth will be appropriate when writing the teacher effectiveness measure.>
42 6. Administration of the Assessment/Performance Task Frequency ofAssessment/PerformanceTaskAdministrationGuiding Questions:How often and when is this assessment/performance task administered?If measuring growth, are multiple assessment windows in place?Sample Statements:Grade 8 General Music: Each of the four project assessments is administered once throughout the course. Formative assessments are inherent in the instructional design. A pretest is administered on the first day of class that assesses prior knowledge applicable to the composition and keyboard assessments.HS Instrumental Music: Students will be assessed at the midpoint of the first and fourth quarters.The administration of the assessment task is viewed from four different perspectives: frequency, resources, adaptations and personnel. The frequency portion is a reminder that formative assessments will inform better instruction and growth statements may require a pretest/post-test design. Either way, teachers should plan up-front to offer formative and summative assessments that will focus instruction on the student learning objective.
43 6. Administration of the Assessment/Performance Task Resources RequiredGuiding Questions:What unique or specific equipment, technologies, or resources are needed to complete this assessment/performance task?Sample Statements:Grade 8 General Music: Keyboards, guitars, audio-playback equipment.HS Instrumental Music: Video/audio recording technology and/or methodologies for students to create online submissions that will require computers and appropriate software.As teachers begin to see more SLO and assessment models, it will be important to make sure that the school district, building and students have the necessary resources to complete tasks successfully. Both commercial and teacher-developed tasks may have financial, technology or equipment needs, so it is relevant to understand what those needs are and how those requirements will be met.
44 6. Administration of the Assessment/Performance Task All Classes: IEP and 504 accommodations will be implemented.Adaptations for Diverse Learners and/or Students with DisabilitiesGuiding Questions:What assessment/performance task adaptations are needed to assist diverse learners and/or students with disabilities?Sample Statements:Grade 8 General Music: Students struggling with keyboard or guitar playing skills can perform simple chord structures.Students can use a computer notation program to create the composition.HS Instrumental Music: IEP adaptations are made (additional time allotted, alternative print formats, teacher-directed instruction to help students feel more comfortable about being assessed individually, etc.)The most obvious adaptations for diverse learners will be found in student IEP and 504 accommodation plans, but teachers may have additional knowledge about students that will encourage unique kinds of support toward student success in achieving the requirements of the assessment tasks.
45 6. Administration of the Assessment/Performance Task Most Classes: The assessments can be administered by an equivalent peer.Some Classes: Student use of equipment needed to complete the authentic assessment could be monitored by an equivalent peer, but should probably be monitored by the class instructor (i.e. athletic, career tech, chemistry, biology equipment).PersonnelGuiding Questions:Can this assessment/performance task be administered by an equivalent peer (educator in a similar content area)? If not, please explain.Does a district policy exist with regard to assessment/performance task administration?Sample Statements:An equivalent peer can administer this assessment. Many students will self-administer this assessment.While Pennsylvania currently has no plan for persons other than the teacher to administer assessment tasks, some states and school districts do. It is recommended that assessment tasks be designed so that an equivalent peer—a teacher of a similar grade or subject—could administer the assessment task. Developing tasks in this manner will alleviate any concerns about a teacher’s inappropriate input into evidence of student achievement, should that be called into question.
46 7. Evidence of Student Achievement Rubrics/ScoringScalesGuiding Questions:How will individual student growth or mastery be determined (defined and scored) using this assessment/performance task? Include the specific rubric/scoring scale that will be used.Does the rubric and/or scoring scale correlate with the assessment/performance task?Sample Statements:HS Instrumental Music:See next slide………….OR LOOK AT SCOTT SHEEHAN’S PMEA WEBINAR PRESENTATIONThank you for your participation in the activity. Please leave all cards on the table so that we can learn about ways to improve the presentation and to improve the SLO template process.The next section of the template works with evidence of student achievement and subsequent strategies and actions to improve instruction and successfully complete the SLO.The Evidence of Student Achievement segments ask questions about scoring the assessment tasks. As you can see from this slide, statements about scoring systems inherent in commercially prepared assessments could be relatively short.
47 7. Evidence of Student Achievement CategoryAdvancedProficientBasicBelow Basic10-98-76-54-0Tone QualityTone quality is controlled, full, rich, and characteristic in all registersTone quality is characteristic in the normal playing range of the instrument, but distorts in a few passages or extreme registers (occasional lapses of control), these lapses rarely hinder the performance.Tone quality exhibits some flaws in production in the normal range of the instrument, extreme registers are usually uncontrolled, tone quality occasionally detracts from performance.Tone quality has significant flaws in basic production in all registers of the instrument and consistently detracts from the performance.IntonationIntonation is accurate in all ranges and registersIntonation is mostly accurate; the few problem pitches are adjusted to an acceptable standard during the performance.Intonation is somewhat accurate but consistently includes out-of-tune notes; these inaccuracies occasionally detract from the performance.Intonation is consistently inaccurate and hinders the quality of performance.Rhythmic AccuracyRhythms are accurate and precise throughout the performance.Rhythms are nearly accurate; occasionally, rhythms lack precise interpretation.Most rhythm patterns are accurate, but errors in precision are present and occasionally detract from the performance.Many rhythms performed incorrectly or inconsistently, major errors are present and detract from the performance.Pitch Accuracy (melodic accuracy)Pitches are consistently correct.Occasional incorrect pitches are played, but do not detract from the performanceMost pitches are correct, but errors are present and occasionally detract from the performance.Many pitches are performed incorrectly; this inaccuracy consistently seriously hinders the performance.As seen in this Grade 8 Art rubric, scoring systems for authentic and teacher-developed assessments could be quite detailed. As with the assessment task statements, principals may appreciate this much detail as they begin to learn about the diverse content areas in which their teachers teach.
48 7. Evidence of Student Achievement Data CollectionGuiding Questions:In what format will data be collected (e.g., database, graphed, portfolio, etc.)Is a pre-post test being used? (If so, please describe.)How frequently will data be collected?Scores from the rubric will be collected on a database. Data will be collected at the mid-point of the first and fourth marking periods.The data collection component is an opportunity to match frequency of task administration with frequency of formative and summative data collection. Additionally, this component looks ahead to ways in which data might be organized to detect changes in student achievement as well as how data will be presented to the principal.
49 7. Evidence of Student Achievement Scoring StudentProgress/PreparationGuiding Questions:How was baseline data collected? (If baseline data was not collected, please explain.)Can baseline data be compared with the results of this assessment/performance task?Sample Statements:Grade 8 General Music: Baseline data is collected through a pretest of retained knowledge; there is no assessment of authentic skills prior to instruction.HS Instrumental Music: Scoring from the first marking period assessment will be used as baseline data for describing growth. Scoring from the fourth marking period assessment will be used to describe growth and/or mastery.The Scoring of Student Progress and Preparation slide is an opportunity to make sure that the scoring procedures are aligned across assessments and relate directly to the assessment task. Commercial systems will probably be well aligned, but teacher-developed rubrics and scoring guides may initially require some careful and focused inspection. As mentioned before, the last slide will provide some websites that demonstrate assessment tasks with connected scoring systems that teachers in many non-tested areas will find useful.
50 7. Evidence of Student Achievement Data PresentationGuiding Questions:What evidence will be presented to principal/evaluator to support the teacher effectiveness measure?How will data be presented to the principal/evaluator (e.g., database, graphed, portfolio, individual student artifacts, etc.)?Sample Statements:Grade 8 General Music: Evidence will be presented through aggregated scores from the database.HS Instrumental Music: An ipad database program created by the teacher will be used to present scoring information.Recognizing that principals and evaluators may not want to have a stack of rubrics presented to them, the SLO template asks teachers how they will present the results of student assessment scoring. Principals may want to see some exemplars of student work, so preparation to keep some artifacts of student work, through video, recordings, projects or portfolios would need to be planned.
51 7. Evidence of Student Achievement All Grades and Courses: The assessment can be scored by an equivalent peer.All Grades and Courses: The assessment is a commercially produced assessment that has a history of validity and reliability.Data Analysis and InterpretationGuiding Questions:How can the assessment/performance task results be interpreted in the same way across equivalent peers?Is there a reliable and valid scoring and interpretive process (i.e., state developed, district-based, commercial, standardized, etc.) that is associated with the assessment/performance task? If so, please describe.Sample Statements:An equivalent peer can use the rubrics to assess the projects.Just as with task administration, some states and school districts are prohibiting the classroom teacher from scoring the assessments of the students that they teach. At present, Pennsylvania has no plan to do this, but individual school districts may. As with task administration, teachers might want to plan as if an equivalent peer—a teacher who teaches the same grade or subject—could score the assessment, regardless if the task and scoring system are teacher-developed or commercially produced.
52 8. Strategies/Actions to Achieve the SLO Assessment for LearningGuiding Questions:What formative assessment information lets you know if your instructional practices will lead to successful completion of the SLO?Sample Statements:Grade 8 General Music: Rubric will be provided to the students as a part of progress monitoring throughout the instructional period.HS Instrumental Music: Interim formative assessments during group lessons will be used to focus instruction toward successful completion of the SLO.While the SLO process is now at a point where the teacher effectiveness measures that we looked at earlier in the presentation can be employed, it is important to remember that improved teaching and learning are the primary reasons to change the way teachers are evaluated. The assessment for learning component guides teachers toward embedding formative processes that will inform them of student achievement along the way to completing the SLO.
53 8. Strategies/Actions to Achieve the SLO Alignment with the Danielson Framework for TeachingGuiding Questions:Based upon reflection, what instructional practices would you like to change or strengthen?What professional learning and/or other type of support will help you to achieve this SLO?Sample Statements:Grade 8 General Music: Based on the formative assessment information provided by student composition and performance tasks, students will be provided with supplemental materials to either address music reading difficulties or enhance current abilities to go beyond the targets. (Danielson 3E)HS Instrumental Music: Based on weekly formative performance assessments, student seating within the band will be uniquely rearranged periodically to provide students the opportunity to develop higher levels of performance independence. (Danielson 2 E)Based on reflective processes found in Domain 4 of the Danielson Framework for Teaching and data gathered from formative assessment processes, teachers should now have very specific information to inform changes in teaching practice or types of professional development that may be needed to achieve the goals of the SLO.
54 SLO Resources Educator Effectiveness: SLO The links listed above are just a few examples of SLO models that can be found in other states. While the Pennsylvania template model is different than models from other states, you will be able to find some well-crafted objectives and teacher effectiveness statements by looking at models from other states. All of the resources listed have models embedded in websites linked to this slide.Assessment Development Resource:beta.ctcurriculum.org
55 The Elective Portion of PA’s Educator Effectiveness Design Student Learning Objectives: Understanding and Completing the SLO TemplateAn Instructional WebinarLink:
56 THANK YOU! Contact: O. David Deitz Educator Effectiveness: SLOContact:O. David DeitzSLO Project Lead, Educator EffectivenessTHANK YOU!Thank you for attending this training to begin your journey toward developing evidences of student achievement that will be applied to your measure of teacher effectiveness. Please feel free to contact O David Deitz with question or concerns, or to submit any SLO models that you would like to have reviewed during this development and pilot period.