Student Learning Objectives (SLOs)

Student Learning Objectives (SLOs)
Dr. Lori Stollar LIU, Division of Educational Services SLO Orientation Module

Session Goals Develop an Understanding of Student Learning Objectives
Develop an Understanding of the SLO process

Norms & Expectations for Today’s Session
Ask Questions Engage Fully Integrate New Information Open Your Mind to Diverse Views Utilize What You Learn

Student Learning Objective
PDE’s Definition: A process to document a measure of educator effectiveness based on student achievement of content standards. Student Learning Objective Concept It is important to keep in mind that the Student Learning Objective (SLO) is a process that allows educators to have a great deal of control and responsibility over their instructional practice and evaluations. In this process, teachers take responsibility for selecting the goals, setting performance indicators/targets, and selecting performance measures which will be used to determine their effectiveness with the students they are instructing in a specific grade level, course, or subject area. These SLOs are one part of the multiple measures that will be used to evaluate teachers and comprises 20% (for teachers with eligible PVAAS data) or 35% (for teachers without eligible PVAAS data) of a teacher’s final rating. Further clarification can be found on the PDE website, Educator Effectiveness Administrative Manual ( Key Points for Trainers Articulate that the student learning objectives are being used in emerging teacher effectiveness systems throughout the nation. They have become increasingly important as teacher effectiveness systems have begun linking student outcomes directly to specific educators, rather than in broad terms associated with school accountability systems. Clarify that SLOs based on non-cognitive measures and used for school effectiveness systems (graduation rates, matriculation rates, AP course enrollment, suspensions, etc.) will be excluded from this SLO process and its usage within educator effectiveness systems. Select non-cognitive measures are part of the Building Level Data. Learning Activity Take three minutes to think: Write down two ways SLOs could positively influence your teaching. Take two minutes to share: Share “Round Robin” in your groups. 4 SLO Orientation Module

4/6/2017 Teachers of Tested Grades and Subjects/Employed under and Instructional I or II and provide direct instruction to students (Domains 1 &3) Goes into effect Region 5

Teachers employed under and Instructional I or II and provide direct instruction to students (Domains 1 &3) Non-tested grades and subjects (eg. K-3, social studies, electives—music, art, technical subject….)

PDE’s Definition: A process to document a measure of educator effectiveness based on student achievement of content standards. Student Learning Objective Concept It is important to keep in mind that the Student Learning Objective (SLO) is a process that allows educators to have a great deal of control and responsibility over their instructional practice and evaluations. In this process, teachers take responsibility for selecting the goals, setting performance indicators/targets, and selecting performance measures which will be used to determine their effectiveness with the students they are instructing in a specific grade level, course, or subject area. These SLOs are one part of the multiple measures that will be used to evaluate teachers and comprises 20% (for teachers with eligible PVAAS data) or 35% (for teachers without eligible PVAAS data) of a teacher’s final rating. Further clarification can be found on the PDE website, Educator Effectiveness Administrative Manual ( Key Points for Trainers Articulate that the student learning objectives are being used in emerging teacher effectiveness systems throughout the nation. They have become increasingly important as teacher effectiveness systems have begun linking student outcomes directly to specific educators, rather than in broad terms associated with school accountability systems. Clarify that SLOs based on non-cognitive measures and used for school effectiveness systems (graduation rates, matriculation rates, AP course enrollment, suspensions, etc.) will be excluded from this SLO process and its usage within educator effectiveness systems. Select non-cognitive measures are part of the Building Level Data. Learning Activity Take three minutes to think: Write down two ways SLOs could positively influence your teaching. Take two minutes to share: Share “Round Robin” in your groups. 7 SLO Orientation Module

Section 1: Classroom Context
General Description Contains demographic information about the educational setting Articulates the course, grade(s), and students on which the SLO is based Provides class size, frequency, and duration data Key Points for Trainers General information about the teacher and class 9 IMT Orientation Draft 02Sept11-CS

Section 2: SLO Goal General Description
Contains a statement about the “enduring understanding” or “big idea” Provides the specific PA standards used in developing SLOs and are the foundation of performance measures. Articulates a rationale providing reasons why the Goal Statement and targeted standards address important learning. Key Points for Trainers Based on the ‘Big Idea’ taken from the PA Standards or Local Curriculum 10 IMT Orientation Draft 02Sept11-CS

Section 3: Performance Measures
General Description Identifies all performance measures, including name, purpose, type, and metric Articulates the administration and scoring details, including the reporting Note: Section 3 is based upon high-quality performance measured aligned to the targeted content standards Key Points for Trainers 5 different types of assessments – Could be Mastery or Growth assessments. 2 required for any Growth measure ** Make sure we address the Performance Task Framework ----- Meeting Notes (10/8/13 14:17) ----- Make sure to look at the blank Performance Task Template 11 IMT Orientation Draft 02Sept11-CS

Section 4: Performance Indicators
General Description Articulates targets the expected level of achievement for each Performance Measure Includes all students in the identified SLO group May include a focused student group Affords opportunity to link indicators and/or weighting Key Points for Trainers This is the target or (Grade) students will work to achieve 12 IMT Orientation Draft 02Sept11-CS

Section 5: Teacher Expectations
General Description Identifies each level (Failing, Needs Improvement, Proficient, Distinguished) students are meeting the PI targets. Reflects an “expectations continuum” established by the educator prior to the evaluation period and then examined at the end of the evaluation period. Selects the overall SLO rating. Key Points for Trainers Teachers predict the % range of students that will reach their targets to determine whether the teacher is proficient. Ex. Set your target first – 80% of my students will reach benchmark to be proficient What is acceptable for a teacher to use for percentages. This is where the administrator will have conversations with teachers on what is an appropriate expectation. Doing for a reason – WE WANT TO SEE GROWTH WITH ALL STUDENTS! 13 IMT Orientation Draft 02Sept11-CS

Guiding Principles SLOs should:
Represent student performance in a specific course/content area taught by the educator. Align to a targeted set of content standards that represent the depth and breadth of the goal statement. Contain results from only high-quality performance measures collected in an equitable, verifiable, and standardized manner. Use metrics based on two time-bound events/data collection periods and/or summative performance with defined levels of achievement. Include performance indicators linked to performance measures. Guiding Principles Concept Keep in mind that there are several guiding principles associated with the SLO process. These principles are key to the SLO process and ensure the development of high-quality, valid, and fair measures of teacher effectiveness through student performance outcomes based on standards. Key Points for Trainers The guiding principles are essential in developing a mental picture of the characteristics of the student learning objective. a. Principle #1-Performance must be aligned to the standards with a specific course/content. Other types of performance, such as attendance, classroom behavior, time-on-task, cooperativeness, engagement, etc. are not part of the SLO process. b. Principle #2-Goal statements articulate a controlling “big idea”; however, more granular content standards are necessary in order to develop/identify performance measures. c. Principle #3-Performance measure quality is critical in producing results associated with the selected standards. Often, assessments are used for purposes the author did not design the tool to perform, thus creating a validity threat to those inferences about student learning. Furthermore, ensure that equitable opportunity for the student to demonstrate learning is provided within the administration and scoring of the performance measure. d. Principle #4-Metrics have strengths and weaknesses. Status [i.e., Mastery] metrics have absolute standards and are easily understood; however, they do not reflect changes (improvement) in student performance Growth metrics are sensitive to changes in learning; however, they are more unstable and limited for high performing students e. Principle #5-Performance indicators provide a clear expectation of performance on the identified assessment. Multiple indicators can often provide a more defensible evidence of student learning. Linking and weighting indicators can be helpful in some instances but add additional layers of complexity to the SLO process. Learning Activity SLO Orientation Module

SLO Process Components
The SLO process contains three (3) phases: Design (ing): thinking, conceptualizing, organizing, discussing, researching Build (ing): selecting, developing, sharing, completing Review (ing): refining, checking, updating, editing, testing, finalizing SLO Process Components Concept The Student Learning Objective (SLO) Process is comprised of three (3) Components: Design, Build, Review. Within the Design(ing) Phase, the trainer will begin to guide the participant in the development of a “Goal Statement”, identifying underlying content standards, and creating a blueprint. The Build(ing) Phase focuses on the completion of the Template #4-SLO Process Template which includes identifying and/or creating performance measures. The Review(ing) Phase allows educators to go back and refine, edit, and finalize the SLO. Key Points for Trainers Explain that all components are done before the school year (initial conversation with principal) in preparing the SLO; however, the REVIEW component may also continue until the final results are available to determine whether or not the performance expectations have been reached. Clarify that specific timelines for the SLO process will be determined by local education agencies (LEAs) and not by the state; however, a generic timeline for the SLO process outlining before, during, and after school year activities should be presented. In general: Teacher develops SLO, along with applicable performance measures before school starts. Principal reviews and discusses with teacher; adjustments may be required. Teacher reviews SLO progress at a midpoint in the year. Principal receives mid-year update from teacher for review. Teacher summarizes performance measure data and evaluates each performance indicator. Teacher presents final SLO results to the principal. Principal assigns final rating in Section 5 of the SLO Process Template. Learning Activity SLO Orientation Module

SLO Process Components DESIGN
Thinking about what content standards to measure Organizing standards and measures Discussing collective goals with colleagues Researching what is needed for a high quality SLO SLO Process Components: DESIGN Concept The SLO Design Process is the “planning” process where teachers work collaboratively to identify targeted standards and types of performance measures that reflect the “big ideas” within standards which would effectively measure student learning and teacher effectiveness. Teachers will work through activities that allow time for thinking, brainstorming, organizing, discussing, and researching content standards, possible performance measures, and performance indicators to be used in the creation of SLOs. The first step in the Design Process is the development of a “goal statement” which reflects the “big ideas” related to the specific standards, along with a rationale statement that explains the alignment to standards and addresses the important learning for the class/course/content area. Training Module 1: Designing will provide further details related to this phase of development. [Note: Teachers will be using Templates #1, #2, and #3 during the Design Phase.] Key Points for Trainers 1. Designing is planning for the SLO, and examining what is needed as well as how performance measures are used to collect information about student achievement. 2. Activities during this stage establish the foundation for developing a student learning objective, including: Identifying target content standards Discussing the “Big Idea” in the standards Thinking about the goal Collaborating with other teachers Brainstorming the type of performance measures to be used Learning Activity Using the SAS portal, have participants review the Curriculum Framework which outlines Pennsylvania’s Big Idea, Concepts, Competencies, and Essential Questions. Have each participant select a specific content area and grade level and share a few of the “big ideas” identified. Using the Homeroom learning portal, have participants review the following materials: (a) Handout #1 – Goal Statement, (b) Handout #2 – Targeted Content Standards, and (c) Handout #3 – Blueprinting. SLO Orientation Module

SLO IN PRACTICE IMT Orientation Draft 02Sept11-CS
HANDOUT – ART SLO EXAMPLE – HELP DESK As we go through the template – Your team will complete as a group ----- Meeting Notes (10/8/13 14:17) ----- As we go through each section of the SLO - Please work together to complete the SLO form IMT Orientation Draft 02Sept11-CS

Section 1: Classroom Context
Element Definition 1a. Name Educator’s full name 1b. School Name of school(s) to which the educator is assigned during the current year. 1c. District Name of district to which the educator is assigned during the current year. 1d. Class/Course Title Name of the class/course upon which the SLO is based. 1e. Grade Level Grade level(s) for those students included within class/course identified in Element 1d. 1f. Total # of Students Aggregate number of students (estimated, across multiple sections) for which data will be collected and applied to this SLO. 1g. Typical Class Size The “average” number of students in a single session of the class/course identified in Element 1d. 1h. Class Frequency The frequency and time frame in which the class/course identified in Element 1d is delivered. 1i. Typical Class Duration The average number of minutes allocated to deliver a “session” of the class/course identified in Element 1d. Key Points for Trainers Participates will complete sections 1 and 2 after review of slides for sections one and two 18 IMT Orientation Draft 02Sept11-CS

Section 1: Art Example IMT Orientation Draft 02Sept11-CS
** Handout –NOTES: The SLO is written for a specific course or class for which that teacher provides instruction. Not for every course or class – A) Not for every class but could be if LEA determines. (Ex. Music teacher working across multiple grade levels. They could choose a class or grade level) 1F - Is the total number of the students in the SLO – Max. 100 – Min. Whatever one class might be. 19 IMT Orientation Draft 02Sept11-CS

What is a Goal Statement?
Definition: Narrative articulating the “big idea” upon which the SLO is based Characteristics: Central to the content area Foundational concept for later subjects/courses What is a Goal Statement? Concept The Goal Statement is integral to the development of an SLO. It is a narrative articulating the “big idea” upon which the SLO is based. The Goal Statement must be aligned with PA standards and/or professional organization standards . These standards can be located at Teachers should reference the following: PA Academic Standards associated with the applicable grade-levels/spans [Note: When no PA Standards are available, use national standards (e.g., World Languages)]. Professional Standards PA Career and Technical Education Programs of Study Key Points for Trainers “Pennsylvania Standards describe what students should know and be able to do; they increase in complexity and sophistication as students progress through school. Using the SAS portal, you can locate specific standards, anchors, and eligible content based on subject area and grade level or course.” (SAS Portal) “Big Idea: Declarative statements that describe concepts that transcend grade levels. Big Ideas are essential to provide focus on specific content for all students.” (SAS Portal) “Essential Questions: Questions connected to the SAS framework and are specifically linked to the “big ideas”. They should frame student inquiry, promote critical thinking, and assist in learning transfer.” (SAS Portal) “Concepts: Describe what students should know (key knowledge) as a result of this instruction specific to grade level.” (SAS Portal) “Competencies: Describe what students should be able to do (key skills) as a result of this instruction, specific to grade level.” (SAS Portal) “Grant Wiggins & Jay McTighe” ‘Understanding by Design’ is a framework for improving student achievement. Emphasizing the teacher’s critical role as a designer of student learning, UbD ᵀᴹ works within the standards-driven curriculum to help teachers clarify learning goals, devise revealing assessments of student understanding, and craft effective and engaging learning activities.” ( Learning Activity 1. Have participants go the SAS portal and review the Curriculum Framework. “The Curriculum Framework specifies what is to be taught for each subject in the curriculum. In Pennsylvania, Curriculum Frameworks include the Big Ideas, Essential Questions, Concepts and Competencies, aligned to Standards and Assessment Anchors and, where applicable, Eligible Content.” (SAS Portal) Module 1-SLO Designing

Section 2: Art Example IMT Orientation Draft 02Sept11-CS ** Handout
NOTES: 21 IMT Orientation Draft 02Sept11-CS

Goal Statements Typically addresses:
WHAT the “big idea” is in the standards HOW the skills and knowledge support future learning WHY the “big idea” is a central, enduring concept (rationale statement) PDE’s SAS portal has identified “big ideas” for most content areas. Goal Statements (Handout #1) Concept The Goal Statement is reflective of the what, why, and how aspects of the SLO being developed. What refers to the specific targeted standards addressed in a particular SLO. What is going to be measured? Why is the rationale for choosing the specific standard. Why is it important to measure this standard? How explains the importance of a specific standard to student learning. How this impacts student learning. Key Points for Trainers Remind participants that Pennsylvania describes “Big Ideas” as declarative statements that describe concepts that transcend grade levels. Big Ideas are essential to provide focus on specific content for all students. (SAS Portal) Remind participants that all information related to PA standards, Big Ideas, Essential Questions, Concepts and Competencies can be found on the SAS Portal. Have teachers work in grade-level or content specific groups. Allow time for reviewing content and/or professional standards. Keep them focused on the “big idea” in a standard. Have them identify why it is important and how it supports student learning. Learning Activity Working in groups, teachers will: Review content standards. Determine the “big idea” in the standard. Discuss the “why” aspect of the statement and reach consensus. Discuss the “how” aspect of the statement and reach consensus. Module 1-SLO Designing

Goal Statement Example
“Apply the concepts and the competencies of nutrition, eating habits, and safe food preparation techniques to overall health and wellness throughout the life cycle at individual, family, and societal levels.” Goal Statement Example (This example comes from Family and Consumer Science/HS Foods and Nutrition Course) Concept Goal Statements should reflect the what, why, and how. What is being measured: Application of the concepts and competencies of nutrition, eating habits, and safe food preparation techniques. Why is it being measured: To determine understanding of the impact to overall health and wellness. How does it impact student learning: Understanding the impact of overall health and wellness impacts on the life cycle at individual, family and societal levels. Key Points for Trainers Review this sample Goal Statement with educator groups. (Refer to the Family and Consumer/Foods Science and Nutrition Standards for 12th Grade at Make sure they can differentiate between the what, why, and how. Learning Activity Module 1-SLO Designing

Your turn With a partner, review the Goal Statement Examples provided. Identify the What, How, and Why of each goal statement.

Goal Statement Examples
Demonstrate understanding of numeric relationships by analyzing and generalizing those relationships using words, graphs, tables, equations and inequalities using technology. (Algebra) The student will demonstrate, through their written works, that audiences differ and that readers’ needs/expectations must be taken into account as one writes. (Language Arts) The student will understand that investigations are conducted to explore new phenomena, check previous results, and to test and compare theories. (Science) Readers will comprehend text by intentionally interacting with it. (Language Arts) Number sentences are able to be modeled by concrete objects and real world scenarios simultaneously. (Elementary Math)

Section 2: SLO Goal Element Definition 2a. Goal Statement
Narrative articulating the “big idea” upon which the SLO is based. 2b. PA Standards References the PA Standards that align with the Goal Statement. Numeric references to PA Standards are found at: References additional professional organization standards that align to the Goal Statement. 2c. Rationale Narrative providing reasons why the Goal Statement and the aligned standards address important learning for this class/course. Standards Must Describe the expectations associated with particular academic content area Articulate what students should know (knowledge) and be able to demonstrate (skills) Be most applicable to student performance outcomes Align with targeted “big idea” or “enduring understanding 26 IMT Orientation Draft 02Sept11-CS

Targeted Standards Choosing Targeted Standards means:
Selecting certain standards for use with the performance measure being developed. Identifying standards which represent the “big ideas” within the content area. Targeted Standards Concept Teachers will be required to select certain standards that align with the “big ideas” within a content area. These targeted standards will become the identified content standards used to create the performance measures. Key Points for Trainers Refer teachers to SLO Guiding Principles (found in the Orientation module, Slide 13) when doing this work. Principle #1 – Represent student performance in a specific course/content area taught by the educator. Steer participants to reference: Pennsylvania Standards : what students should know and be able to do Pennsylvania Curriculum Frameworks: descriptions of overarching performance goals (with their related understandings and questions) that shape how the content standards are organized and used to guide teaching and assessment. (Wiggins & McTighe; “Schooling By Design,” p. 74) Learning Activity Refer teachers to the SAS Portal where they can find the Pennsylvania Standards and the Curriculum Framework. a. Pennsylvania Standards describe what students should know and be able to do. The SAS Portal allows teachers to locate specific standards, anchors, and eligible content based on subject area and grade-level or course. (e.g., Mathematics, Standard Area – 2.1: Numbers, Number Systems and Number Relationships, Grade Level – 2.1.3: Grade 3, Standard – A: Apply one-to-one correspondence and number patterns to count up and count back and to compare values of whole numbers and values of money) b. Pennsylvania Curriculum Framework describes overarching performance goals. It includes the Big Ideas, Essential Questions, Concepts and Competencies aligned to Standards and Assessment Anchors and, where appropriate, Eligible Content. (SAS Portal) Module 1-SLO Designing

Targeted Standards Criteria
Are a refined list of the content standards. Represent the essential knowledge and skills that students are expected to acquire. Are the standards upon which educators will spend the most time. Create transparency for families and the community about what is most important for student success. Are the identified content standards used to create the performance measures. Targeted Standards Criteria Concept Since these targeted standards will be used to create the performance measures, this will be a refined list and not an all-inclusive list. This refined list will represent the knowledge and skills that students are expected to acquire and upon which educators will spend the most time. Working in groups, teachers will: Review the Goal Statement; Review the “what”, “why”, and “how” of the statement; Review content standards and/or big ideas/enduring understandings; Identify targeted set of standards and/or big ideas/enduring understandings to narrow focus; and, Compare targeted standards and/or big ideas/enduring understandings to Goal Statement. Ensure alignment and indicate what is going to be emphasized on the performance measure. Key Points for Trainers Refer teachers to SLO Guiding Principles (Orientation Module, Slide 13) when selecting targeted standards. a. Principle #2 – “Align to a targeted set of content standards that represent the depth and breadth of the goal statement.” Goal statements articulate a controlling “big idea”; however, more granular content standards are necessary in order to develop/identify performance measures. Learning Activity Module 1-SLO Designing

Guiding Questions ENDURANCE- Will this standard provide students with knowledge and skills of value beyond a single test date? LEVERAGE- Does this standard provide knowledge and skills of value in multiple disciplines? READINESS FOR THE NEXT LEVEL OF LEARNING- Will this standard provide students with essential knowledge and skills necessary for success in the next level of instruction? Guiding Questions Concept When teachers are choosing the targeted standards, they should use the three (3) guiding questions on this slide as a measure to determine the effectiveness of the standard chosen. If they can answer “yes” to all three of these questions, they likely identified a meaningful content standard that will provide a strong basis for building the performance measures. Key Points for Trainers The SAS Portal provides Big Ideas aligned to the Pennsylvania Standards. When reviewing the Curriculum Framework, teachers can choose the grade level and content area. Once they do that, they will be able to see the “Big Ideas” associated with that content area and grade level. They can then click on a particular “big idea” and it will show them the essential questions, concepts, competencies, standards and eligible content related to the Big Idea. 2. Example using Mathematics/Third Grade- Big Idea – The base-ten number system is a way to organize, represent, and compare numbers using groups of ten and place value. Essential Question – What strategies and models help us understand how to solve multiplication and division problems and how multiplication and division are related/connected? Concepts – fractions and decimals: develop basic understandings Competencies – develop an understanding of multiplication and division of whole numbers by building on knowledge of the base ten system/operations and through the use of representations (e.g., equal-sized groups, arrays, area models, equal “jumps” on number lines, successive subtraction, partitioning, and sharing) in order to describe relationships, make estimations, and solve problems. Standards/Eligible Content – B, D, A, D, A, B, C, D… (SAS Portal) Learning Activity Module 1-SLO Designing

Targeted Standards Example
Standard Id Description Rationale ACTFL 1.1 WL 1.1—Students listen and respond, engage in conversations, provide and obtain information, express preferences, emotions and feelings, and exchange opinions and beliefs. (Interpersonal) To achieve functional levels of communicative competence in a world language, students need to use the language regularly in everyday social interactions such as conversing, arguing, criticizing, requesting, convincing and explaining effectively. ACTFL 1.2 WL 1.2—Students comprehend and interpret written and oral language on a variety of topics. (Interpretive) Developing literacy in a world language is a crucial 21st century skill. Students need to develop a variety of reading and listening strategies that will allow them to comprehend, analyze and synthesize information. ACTFL 1.3 WL 1.3—Students present information, concepts and ideas in oral and written form on a variety of topics. (Presentational) Students need the ability to gather, comprehend, evaluate, synthesize and report information and ideas. The need to conduct research and consume media intended for speakers of other languages are crucial 21st Century skills. Targeted Standards Example (Template #2) (Note: This example comes from World Language. A demonstration of how to use national standards is shown in this slide; however, PA Standards should always be given first and primary consideration.) Concept This slide illustrates how a completed Template #2 should look. It includes the specific targeted standards that align with the “big idea” outlined in the Goal Statement. It also provides a description and rationale associated with each targeted standard. (Some teachers will be familiar with the term “big idea.” This term can be considered synonymous with the term “enduring understanding.”) Key Points for Trainers 1. Standard Id ACTFL 1.1 – this is the code associated with each specific standard found within the Pennsylvania content standards, in this case, the standards are national standards of the ACTFL. 2. Description WL Students listen and respond, engage in conversations, provide and obtain information, express preferences, emotions and feelings, and exchange opinions and beliefs. (Interpersonal) This column denotes the specific expectations associated with a particular standard area. 3. Rationale To achieve functional levels of communicative competence in a world language, students need to use the language regularly in everyday social interactions such as conversing, arguing, criticizing, requesting, convincing and explaining effectively. The rationale explains why this standard is important to student learning and how it aligns with the “big idea.” Learning Activity Review Handout #2 with educator groups and make sure they understand what each column denotes. If necessary, show the audience where the Standard Id’s are located within the Curriculum Frameworks. Module 1-SLO Designing

Your Turn Goal Statement Template Indicate goal statement
Use big ideas from SAS or district curriculum to complete Goal Statement and Rationale Indicate goal statement Given the goal statement, identify the underlying content standards

SLO Process Components BUILD
Selecting the performance measure(s) Developing targets and expectations Completing the template Sharing the draft materials with other colleagues SLO Process Components: BUILD Concept Within the Build Phase, teachers will be completing the SLO Process Template which identifies the Classroom Context, SLO Goal, Performance Indicators (PI), and Performance Measures (PM). When selecting or developing Performance Measures, the Assessment Literacy Series (ALS) will guide educators to ensure that the measures used meet the principles of well-developed measures. Well-developed measures must: Be built to achieve the designed purpose; Produce results that are used for the intended purpose; Align to targeted content standards; Contain a balance between depth and breadth of targeted content; Be standardized, rigorous, and fair; Be sensitive to testing time and objectivity; and, Have score validity and reliability evidence. Training Module 2: Building will provide further details related to this phase of development. [Note: Teachers will be using Templates #4 and #5 during the Build Phase.] Key Points for Trainers Ensure the participants understand that Building is an iterative process between the original design and the creation of the SLO. Often, the original design must be changed after it is decided how standards will be measured and performance indicator targets have been developed. Activities during this stage [complete the SLO Process Template] include: Selecting (or creating) the performance measures that are aligned to the targeted content standards; Developing mastery and/or growth metrics associated with the performance measures; Establishing performance indicator targets; Identifying students included in the SLO data; and, Creating performance expectations. Learning Activity Using the SAS portal and the Homeroom learning portal, have participants review the following: (a) Template #4 SLO Process Template, (b) Template #5 Performance Task Framework, and (c) Model #1 (Art) and Model #2 (Physical Education). Have participants identify and explain each section of Template #4 SLO Process Template. (A list of “Help Desk” statements designed to support this activity can be found in SLO/Build/Other Stuff.) SLO Orientation Module

General Description Identifies all performance measures, including name, purpose, type, and metric Articulates the administration and scoring details, including the reporting Note: Section 3 is based upon high-quality performance measures aligned to the targeted content standards (see Assessment Literacy Series-ALS materials) Section 4: Performance Measures Concept Each section of the SLO Process has a specific task and each has its own importance. However, Section 4: Performance Measures is the most critical and probably the most challenging work to complete. Selecting and/or developing Performance Measures that are of high-quality is essential for demonstrating student achievement of the selected content standards. Key Points for Trainers Make sure the participants understand that performance measures must allow equitable opportunities for students to demonstrate learning. Discuss the strengths and weaknesses of status (mastery) and growth metrics. a. Status metrics have absolute standards and are easily understood; however, they do not reflect changes (improvement) in student learning. b. Growth metrics are sensitive to changes in learning; however, they are more unstable and can be limited for high performing students. Principles of Well-Developed Measures: Be built to achieve the designed purpose Produce results that are used for the intended purpose Align to targeted content standards Contain a balance between depth and breadth of targeted content Be standardized, rigorous, and fair Be sensitive to testing time and objectivity Be valid and reliable Learning Activity Module 2-SLO Building

Principles of Well-Developed Measures
Measures must: Be built to achieve the designed purpose Produce results that are used for the intended purpose Align to targeted content standards Contain a balance between depth and breadth of targeted content Be standardized, rigorous, and fair Be sensitive to testing time and objectivity Have score validity and reliability evidence Principles of Well-Developed Measures Concept Working through the steps associated with the Assessment Life Cycle focuses test developers on the principles of well-developed measures. The Performance Measure Rubric for Teachers (found in ALS/Review/Stuff) will guide participants through the process of selecting/developing high-quality performance measures. The rubric has 18 technical aspects related to the basic principles of quality assessment. They are organized into three (3) strands to align with the Design, Build, Review components of both the SLO Process and the ALS Process. Using this rubric allows teachers to validate the performance measures they have determined will effectively measure student progress toward the goals identified. Key Points for Trainers Measures must: Be built to achieve the designed purpose – “For all types of assessments, the first step is to clearly define the purpose. Teachers must specify exactly what the assessment is intended to measure, characteristics of intended test takers, types of scores to be reported, and how the information derived from the assessment will be used.” (RIA “Smart Book”, 2011) “If the purpose is not well defined, there is a high risk that the assessment will not satisfy fundamental measures of test validity. Critical measures of test validity examine whether the test is built to achieve its purpose(s) and whether the results are used for the intended purpose(s). (RIA, “Smart Book”, 2011). b. Strand 1: Design of the Performance Measure Rubric Produce the results that are used for the intended purpose - does the assessment measure what we really want to measure? Evaluate the final assessment against the intended purpose. i The purpose of the performance measure is explicitly stated (who, what, why) a. Strand 2: Build b. Strand 3: Review i Item/tasks are created and reviewed in terms of: (a) alignment to the targeted standards, (b) content accuracy, (c) developmental appropriateness, (d) cognitive demand, and (e) bias, sensitivity, and fairness. i The performance measures are reviewed in terms of design fidelity- Items/tasks and forms are rigorous and free of bias, sensitive, or unfair characteristics. Item/task and form statistics are used to examine levels of difficulty, complexity, distracter quality, and other properties. Items/tasks are distributed based upon the design properties found within the specification or blueprint documents. Aligned to targeted content standards - “The desired attributes and characteristics of the assessment need to be identified, specified, and documented. The test design framework and blueprint provide information necessary to guide the item/task development process. This is a critical activity in establishing validity.” (RIA “Smart Book”, 2011) a. Strand 1: Design of the Performance Measure Rubric 1.4 Specification tables articulate the number of items/tasks, item/task types, passage readability, and other information about the performance measure – OR – blueprints are used to align items/tasks to targeted content standards. 1.2 The performance measure has targeted content standards representing a range of knowledge and skills students are expected to know and demonstrate. i The performance measures are reviewed in terms of alignment characteristics- Pattern consistency (within specifications and/or blueprints Developmental appropriateness Cognitive demand Matching the targeted content standards Contain a balance between depth and breadth of targeted content – “Several considerations are taken into account during the assessment design phase. First, to satisfy accepted standards of reliability and validity, a minimum number of items and score points are required within each subtest and for the overall assessment. Second, the assessment must include items with a range of difficulty levels if the assessment’s purpose is to provide information about student achievement at different levels on the performance continuum.” (RIA, “Smart Book”, 2011) 1.5 Items/tasks are rigorous (designed to measure a range of cognitive demands/higher-order thinking skills at developmentally appropriate levels) and of sufficient quantities to measure the depth and breadth of the targeted content standards. Be standardized, rigorous, and fair – “A number of technical and editorial issues are related to item and test specifications. Obviously, crucial quality processes are needed in assessment development, such as adequate content coverage and the development of items and test forms to meet best practice requirements. The desired psychometric properties for the items, such as difficulty and discrimination, as well as desired test properties such as overall test difficulty and reliability need to be targeted and met. Detailed item specifications must include: item types, number of items, response options, difficulty levels, language load, and artwork. Test specifications guide the process of developing forms and likewise need to be detailed to address issues of content coverage, overall difficulty level, balance of items, targeted distribution of item difficulties, rules for sequencing items, total number of items, and timing.” (RIA “Smart Book”, 2011) b. Strand 2: Build i. 1.3 The performance measure’s design is appropriate for the intended audience and reflects challenging material needed to develop higher-order thinking skills. 2.1 Items/tasks and score keys are developed using standardized procedures, including scoring rubrics for human-scored, open-ended questions (i.e., short constructed response, writing prompts, performance tasks, etc.). 2.2 Item/tasks are created and reviewed in terms of: (a) alignment to the targeted standards, (b) content accuracy, (c) developmental appropriateness, (d) cognitive demand, and (3) bias, sensitivity, and fairness. 2.3 Administrative guidelines are developed that contain the step-by-step procedures used to administer the performance measure in a consistent manner, including scripts to orally communicate directions to students, day and time constraints, and allowable accommodations/adaptations. 2.4 Scoring guidelines are developed for human-scored items/tasks to promote score consistency across items/tasks and among different scorers. These guidelines articulate point values for each item/task used to combine results into an overall score. a. Strand 3: Review 3.2 The performance measures are reviewed in terms of editorial soundness, while ensuring consistency and accuracy of other documents (i.e., administration)- Applies Universal Design principles Identifies words, text, reading passages, and/or graphics that require copyright permission or acknowledgement Be sensitive to testing time and objectivity – Ensures linguistic demands and/or readability is developmentally appropriate Strand 2: Build 2.6 The total time to administer the performance measure is developmentally appropriate for the test-taker. Generally, this is 30 minutes or less for young students and up to 60 minutes per session for older students (high school). Have score validity and reliability evidence – “General scoring criteria and methods for scoring both short-constructed response (SCR) and extended-constructed response (ECR) should be outlined when the guidelines for writing items are developed.” i Summary scores are reported using both raw score points and performance level. Performance levels reflect the range of scores possible on the assessment and use terms or symbols to denote performance levels. 3.5 As part of the assessment cycle, post administration analyses are conducted to examine such aspects as items/tasks performance, scale functioning, overall score distribution, rater drift, content alignment, etc. 3.4 Cut scores are established for each performance level. Performance level descriptors describe the achievement continuum using content-based competencies for each assessed content area. 3.6 The performance measure has score validity evidence that demonstrated item responses were consistent with content specifications. Data suggest the scores represent the intended construct by using an adequate sample of items/tasks within the targeted content standards. Other sources of validity evidence such as the interrelationship of items/tasks and alignment characteristics of the performance measure are collected. 3.7 Reliability coefficients are reported for the performance measure, which includes estimating internal consistency. Standard errors are reported for summary scores. When applicable, other reliability statistics such as classification accuracy, rater reliabilities, and others are calculated and reviewed. Learning Activity Module 2-SLO Building

Formerly Section 4 Element Definition 4a. Name (5 entry spaces are provided throughout Section 4, but 5 are not required) List the name of each Performance Measure. 4b. Type Identify the type(s) of Performance Measure(s). From the given list, select all types that are applicable. 4c. Purpose The purpose statement for each Performance Measure that addresses who, what, why. 4d. Metric The metric used by the Performance Measure to evaluate the Performance Indicator. 4e. Administration Frequency The timeframe during the school year that the Performance Measures are administered to students. For Performance Measures administered more than one time, the frequency (e.g., quarterly) is annotated. 4f. Adaptation / Accommodations Identifies and lists any unique adaptations or special accommodations needed for IEP, ELL, Gifted IEP, or Others to complete the tasks within each Performance Measure. Key Points for Trainers 3535 IMT Orientation Draft 02Sept11-CS

Section 3: Art Example Formerly Section 4
36 IMT Orientation Draft 02Sept11-CS

** Handout – Art SLO Handout NOTES: The SLO is written for a specific course or class for which that teacher provides instruction. Not for every course or class – A) Not for every class but could be if LEA determines. Write an SLO for the course that you have the most students. (Ex. Music teacher working across multiple grade levels – the SLO can be written for one grade level. SLO needs to be aligned to specific content standards 1.F – is the total of the students in the SLO – Max. 100 – Min. Whatever one class might be. 37 IMT Orientation Draft 02Sept11-CS

Formerly Section 3 General Description Articulates targets for each Performance Measure Includes all students in the identified SLO group May include a focused student group Affords opportunity to link and/or weight indicators Section 3: Performance Indicators Concept Performance indicators are a description of the expected level of achievement on each measure used in the SLO. An understanding of the scoring tool used to describe student achievement for any given performance measure is necessary to write a performance indicator statement. Sample performance indicator statements, as found in the “Help Desk”, are listed below: Physics (1) Roller Coaster Energy Project (2) achieve 6 out of 9 using the roller coaster project rubric. US History (1) US History Final Exam (2) achieve an 85% or higher on the final exam. 5th Grade ELA (1) DRA (2) Using the DRA text gradient chart, one year of reading growth. Referring to SLO Design/Template #3 (SLO Blueprint), teachers can begin to fill out Section 3: Performance Indicators of the SLO Process Template. When teachers filled out the SLO Blueprint, they identified the targets for each Performance Measure selected and/or to be developed. Key Points for Trainers Remind teachers that Performance Indicators offer a great deal of flexibility in the system. Performance Indicators can be linked – meaning a student must meet a specific achievement level across two or more Performance Measures in order to meet the standard. Also, Performance Indicators can be weighted if there is more than one (1) Performance Indicator. Performance Indicators should be specific, measurable, and ambitious, but attainable. The “Help Desk” definition document provides further details and examples of linked and weighted Performance Indicators. Learning Activity Module 2-SLO Building

Formerly Section 3 Element Definition 3a. Performance Indicator (PI) Targets: All Student Group (5 entry spaces are provided, but 5 are not required) A description of the expected level of achievement for each student in the SLO population (as defined in Element 1f) based on the scoring tool(s) used for each Performance Measure (as listed in Element 4a). 3b. Performance Indicator (PI) Targets: Focused Student Group (optional) (5 entry spaces are provided, but 5 are not required) A description of the expected level of achievement for each student in a subset of the SLO population (as defined in Element 1F) based on the scoring tool(s) used for each Performance Measure (as listed in Element 4a). Subset populations can be identified through prior student achievement data or through content-specific pre-test data. 3c. PI Linked A description of any Performance Measures for which a student must meet a specific achievement level in order to meet achievement levels on additional Performance Measures. 3d. PI Weighting (optional) An assignment of proportional values among PIs prior to aggregation and application to Section 5. Weighting can be applied when there is more than one Performance Indicator. Key Points for Trainers Look at 3a and 4a and ask participants how they are connected. 4b and 4d Use 4b and 4d to determine which measures will connect best to your goal statement. Complete 4a and 3 a 3939 IMT Orientation Draft 02Sept11-CS

40 IMT Orientation Draft 02Sept11-CS

Your Turn Based on your Goal Statement and Content Standards, determine Performance Measures that will measure your goal. Determine the desired targets (or Performance Indicators) for each Performance Measure.

General Description Classifies percentages of students who are meeting the Performance Indicator targets into four levels: Failing, Needs Improvement, Proficient, and Distinguished. Selects the overall Elective rating. Section 5: Teacher Expectations Concept As part of the SLO Process, educators will be required to set expected levels of student learning based on the Performance Indicators and their Performance Measures. The levels are identified as “Distinguished, Proficient, Needs Improvement, and Failing.” This Teacher Expectation Rating will determine the overall SLO rating which will be used as the Elective Rating on the Pennsylvania Educator Effectiveness Rating Tool. Key Points for Trainers 1. Participants learning the SLO process often confuse and intertwine the Performance Indicator Statement (3a) with the Teacher Expectation Statement (5a), especially when the Performance Indicator statement is described in percentages. Examples: Performance Indicator Statement: achieves at minimum an 80% on the post-test. Teacher Expectation Statement: 80% of students meet the performance indicator (implies that 80% of students achieve at minimum an 80% on the test). The SLO Process allows for a great deal of control and flexibility over the performance measures and the expected levels of growth and/or mastery associated with each measure that will be used to fulfill the requirements of the Elective portion of teacher evaluation. These levels are established by educators prior to the evaluation period and each performance level is populated with a percentage range such that 0% to 100% meeting expectations is distributed among the levels. The Elective Rating for teachers is not completed until after performance data are collected, reviewed, and evaluated against each Performance Indicator. Learning Activity Module 2-SLO Building

Element Definition 5a. Level Four levels of projected performance regarding the PI, reflecting a continuum established by the educator prior to the evaluation period. Each performance level (i.e., Failing, Needs Improvement, Proficient, and Distinguished) is populated with a percentage range such that 0% to 100% meeting expectations is distributed among the levels. 5b. Elective Rating Given the actual performance regarding the PI, the principal or evaluator identifies one of four performance levels. This section is not completed until after performance data are collected, reviewed, and evaluated against each Performance Indicator, and in the aggregate, against 5a criteria. Notes/ Explanation Provides space for the educator to articulate influences, factors, and other conditions associated with the assigned rating as well as to reflect on purposeful review of the data. Key Points for Trainers 43 IMT Orientation Draft 02Sept11-CS

Section 5: Art Example IMT Orientation Draft 02Sept11-CS
----- Meeting Notes (10/8/13 15:05) ----- We need to have examples to share with groups. 44 IMT Orientation Draft 02Sept11-CS

Section 5: Teacher Expectations (cont.)
Independent Performance Indicators SLO based on 100 students in each indicator PI #1 Jumping PI #2 Hopping PI #3 Sprinting Meet Expectations 25 50 75 Total Number of Students 100 Section 5: Teacher Expectations (continued) Concept This slide provides one (1) example of how Teacher Expectations levels can be determined. Key Points for Trainers Once the performance data has been collected, reviewed, and evaluated against each Performance Indicator, the number of students meeting expectations can be charted and totaled. Then an overall percentage can be determined by a simple division problem: the total number of students meeting expectations/the total number of students in the SLO population across all indicators. The same student may be included across all indicators (which is demonstrated in the slide); however, some variation in student counts across time will exist. Remember, Teacher Expectations and Performance Indicators are two different things: Performance Indicators are the descriptions of the expected level of achievement for each student in the SLO population on a particular Performance Measure – Physical Education/3rd grade Model Teacher Expectations – are the four levels of projected performance regarding the PI, reflecting a continuum established by the educator prior to the evaluation period. Each performance level (i.e., Failing, Needs Improvement, Proficient, and Distinguished) is populated with a percentage range such that 0% to 100% meeting expectations is distributed among the levels. – Physical Education/3rd grade Model Failing – 0% - 60% of students will meet the PI targets. Needs Improvement – 61% to 84% of students will meet the PI targets. Proficient – 85% - 94% of students will meet the PI targets. Distinguished - 95% - 100% of students will meet the PI targets Another problematic area is where to set the values among the different performance levels. A “trade-off” exists between the rigor of the performance targets and expectation of teachers that students will meet those standards. In looking at the Physical Education – 3rd grade Model, the different PI Targets for the focused student group is reflective of the “trade-off” between the rigor of the performance targets and expectation of teachers that students will meet those standards. Because the PI Targets are based on both Mastery and Growth – the teacher expectations are different for different students. Learning Activity 1. Using the Physical Education-3rd grade Model, have participants create student results and from that data determine Section 5: Teacher Expectations The sum of all students meeting expectations ( ) The sum of all students ( ) Resultant: (150/300 = .50 or 50%) Module 2-SLO Building

GOAL STATEMENT RATING PERFORMANCE INDICATOR PERFORMANCE MEASURE ALL STUDENTS TARGETED STUDENTS SLO Design Coherency Concept This figure is designed to help visualize how the “parts” fit into the “whole.” Key Points for Trainers It is essential for the audience to fully understand how only a coherent SLO approach should be used within the greater Educator Effectiveness System. a. Goal Statements are the “big idea” upon which the SLO is based. These goals must be based upon PA standards and provide a rationale for choosing a particular standard which addresses the important student learning. b. Performance Indicators are the expected levels of achievement for students in the SLO population based on the performance measures. c. Performance Measures are the various tools/assessments which will be used to measure student achievement toward a specific goal. d. Ratings are determined by the overall results given the expectation ranges established at the beginning of the year. Learning Activity 1. Have participants go to the SAS portal and review the Curriculum Framework. “The Curriculum Framework specifies what is to be taught for each subject in the curriculum. In Pennsylvania, Curriculum Frameworks include Big Ideas, Concepts, Competencies, and Essential Questions aligned to Standards and Assessment Anchors and, where appropriate, Eligible Content.” (SAS Portal) SLO Orientation Module

SLO Process Components REVIEW
Checking the drafted SLO (including the performance measures) for quality Refining measures and targets Editing text and preparing discussion points/highlights for principal Finalizing materials Updating completed SLO with performance data SLO Components: REVIEW Concept The SLO Review Process provides an opportunity for educators to complete a 3-tier Quality Assurance Review to ensure the SLO, along with its applicable performance measures, are complete, comprehensive, and coherent. Teachers will be using checklists and rubrics to determine the Completeness of the SLO Template, the comprehensiveness of the performance measures, and the coherency (alignment) of the SLO. Training Module 3: Reviewing will provide further details related to this phase of the process. [Note: Teachers will use Handouts #4 and #5, and Template #6 during the Review Phase.] Key Points for Trainers Ensure the participants understand that Reviewing requires an extensive evaluation of the SLO’s quality in terms of the 3C’s. Completeness, Comprehensiveness, and Coherency Quality assurance checklist and rubric Process activities during this stage occur before and after the presentation to the principal, and include: Finalizing and submitting the proposed SLO; Refining the SLO based upon feedback from the principal; Collecting performance data on student achievement; Monitoring the SLO during the school year; Updating the SLO with data; Evaluating each performance indicator; and, Determining the Elective Rating. Learning Activity Allow time for participants to access and review the following documents from the Homeroom learning portal: a. Handout #4 – Quality Assurance Checklist-SLO b. Handout #5 – SLO Procedural “Cheat Sheet” c. Template #6 – SLO Coherency Rubric d. Refinement Control Checklist e. Performance Measure Rubric for Teachers SLO Orientation Module

Training Resources SAS is the PDE website (www.pdesas.org) containing:
Pennsylvania content standards and other helpful PDE developed material a downloadable SLO training “packet”, including SLO Models links to Research in Action’s training platform, Homeroom Training Resources Concept As noted previously, a significant number of resources are available on the above mentioned sites. Go to the SAS portal at for PA content standards and PDE developed material. Also on the SAS portal is a link to RIA’s Homeroom portal. Both sites provide trainings and participants with any information needed for the SLO process. Note: the icons provided on the slide are hyper-linked. Key Points for Trainers Built to support teachers and principals. Build template on-line, save, edit, print. View training at a time that “fits” in to user’s schedule (resources are available online). Learning Activity Pause training and have audience go to the SAS Portal and locate (a) a content-specific curriculum framework, and (b) the SLO “Homeroom” webpage. (New participants should register on the RIA website when the Login page appears.) 2. As preparation for the next slide, participants should click on the SLO box and the ALS box to gain familiarity with the visual presentation. Note: These web resources will assist in completing templates in the Design module. Homeroom is RIA’s web-based learning platform ( containing: on-line training materials, including the SLO Process Template downloadable SLO training files, except the videos links to the SAS portal SLO Orientation Module

Tool Organization Videos Guides Templates “Other Stuff”
Describe the procedures within each of the three phases (i.e., Design, Build, & Review) Guides Provide examples and information about a process Templates Assist in developing customized material “Other Stuff” Supplements the core training materials Tool Organization Concept As you work through the various training modules, several resources are available. All SLO training materials can be found at both the PDE’s SAS portal and Research in Action’s Homeroom learning platform. Go to Once there you can login to the Homeroom learning portal. Key Points for Trainers Differentiate how tools are organized into four groups: videos, guides, templates, and “other stuff”. a. Videos are PowerPoint, mp3, etc. that provide information in a visual format. b. Guides have materials such as handouts, rules of thumb, model SLOs, etc. that reinforce content presented in the videos. They contain the details within each element of the process. c. Templates are used to complete each phase of the process. They create a structure for creating learning objectives and performance measures. [Note: Performance measure templates are located within the Assessment Literacy Series found within the Homeroom learning portal]. During the Design Phase, teachers will complete Template #1 Goal Statement, Template #2 Targeted Content Standards, and Template #3 SLO Blueprint During the Build Phase, teachers will complete Template #4 SLO Process Template and Template #5 Performance Task Framework During the Review Phase, teachers will complete Template #6 SLO Coherency Rubric d. “Other Stuff” is where additional materials, references, and information reside that may be helpful to the process. Learning Activity 1. Have participants go to to review resources that are available on the SAS portal and the Homeroom learning portal. SLO Orientation Module

SLO Design, Build, Review: Using PDE’s Online Tools to Implement the SLO Process SAS Portal:

Navigate to the homeroom page: RIA Homeroom site.

Log in and if not a user then register for the site: Pause until entire room is registered or with a partner:

Home Page for information:
Open SLOs

The SLO Box expands…………..

Joining the SLO Professional Learning Community on SAS.
Go to the SAS home page( Log in with your user name and password. If you do not have an account with SAS you will have to create one.

Enter your information on the log in page and submit.

Once you have successfully logged in and are at the SAS home page, go to Teacher Tools in the upper right corner.

Click on Teacher Tools, this will provide you with various tools.
Locate the button labeled “My Communities.”

This will open your membership to various Professional Learning Communities.
If you are not a member of the Student Learning Objectives PLC, type SLO in the search bar.

Once a member of the SLO community you will have access to communication with all other members and a calendar of upcoming events.

(This is located at the bottom of the SLO community page.)
Along with posting questions to the entire community you have access to the Digital Repository, in which SLO training materials and supporting documents are located. (This is located at the bottom of the SLO community page.)

Student Learning Objectives (SLOs)

Similar presentations

Presentation on theme: "Student Learning Objectives (SLOs)"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Student Learning Objectives (SLOs)

Similar presentations

Presentation on theme: "Student Learning Objectives (SLOs)"— Presentation transcript:

Similar presentations

About project

Feedback