Presentation is loading. Please wait.

Presentation is loading. Please wait.

Understanding and Using Data to Plan for School Improvement

Similar presentations


Presentation on theme: "Understanding and Using Data to Plan for School Improvement"— Presentation transcript:

1 Understanding and Using Data to Plan for School Improvement
2009 Reading and Mathematics Assessment for Learning

2 Outcomes Bringing context and meaning to the math and reading assessment project results; Initiating reflection and discussion among school staff members related to the math and reading assessment results; Encouraging school personnel to judiciously review and utilize different comparators (criterion-referenced, experience-referenced, standards-referenced, longitudinal-referenced, and normative-referenced) when judging assessment results; Modeling processes that can be used at the school level for building understanding of the links between assessment, curriculum and instruction among school staff and the broader community; and, Providing opportunity to discuss and plan at the school level

3 Agenda Principles of the Saskatchewan Assessment for Learning Program
Conceptual Frameworks Nature of Assessment Data Comparators Opportunity-to-Learn Student Performance Outcomes Standards and Cut Scores Mining the Report Data Analysis Designing Interventions Identifying Teaching and Learning Strategies Action Planning

4 Purposes of the AFL Project
Stimulate and inform discussion around student performance Assist identification of areas of focus for school and division planning in reading and mathematics Promote and support a teaching and learning community across grade levels Strengthen assessment literacy and use of data for improvement.

5 Principles of the AFL Program
Cooperation and Shared Responsibility Equity and Fairness Comprehensiveness Continuous Improvement that Promotes Quality and Excellence Teacher Professionalism Authenticity and Validity Honesty and Openness 1. Cooperation and Shared Responsibility - All participants work together to formulate assessment programs, standards and reporting approaches. There is shared responsibility at the classroom, school, division and provincial levels for reliable and timely accountability measures. 2. Equity and Fairness - Evaluation practices recognize that the determinants of educational outcomes vary greatly among classrooms, schools, and communities. Procedures are sensitive to sociodemographic differences such as economic status, culture, gender and geographic location. 3. Comprehensiveness - Assessment and evaluation practices endorse a broad range of performance indicators (including process, context and outcome indicators) to provide a holistic and balanced picture of system performance as well as of student learning and experience. 4. Continuous Improvement that Promotes Quality and Excellence - We are committed to gathering evaluative information and providing supportive feedback of results to enable continuous improvement throughout the education system. 5. Teacher Professionalism - Evaluation activities empower teachers by respecting and reflecting on their central role in education and contributing to their professionalism and continuous professional development. Self-evaluation and reflection on practice with a view to continuous improvement are part of professional practice. 6. Authenticity and Validity - All activities are designed to facilitate authentic assessment (e.g., connected to curriculum objectives, non-biased, performance based). The information gathered in an assessment meets the needs of intended users – students, teachers, administrators and the Department. All assessment practices adhere to the Fair Student Assessment Practices for Education in Canada and to legislation concerning freedom of information and privacy. 7. Honesty and Openness - All assessment procedures and instruments are open for scrutiny, with evidence used in drawing conclusions and recommendations freely available. Both strengths and weaknesses are reported and all evaluative practices are transparent.

6 Conceptual Frameworks: pps 3 & 4
With a partner, read the Conceptual Framework for either Math or Reading. Use the Say Something strategy as you read. Determine with your partner how you will break up the document into sections. Pause after reading each section, look up and say something – an aha, a question or a comment. The guiding principles of the provincial AFL program have been agreed upon by all education partners in Saskatchewan.

7 Nature of Assessment Data
Definitive Indicative Individual Classroom School Division Provincial National International Student Evaluations System Evaluations From Understanding the numbers. Saskatchewan Learning

8 Depth and Specificity of Knowledge
. In-depth knowledge of specific students Little knowledge of specific students In-depth knowledge of Individual National School Classroom International Division Provincial Assessments In-depth knowledge of systems Individual National School Classroom International Division Provincial Assessments From Saskatchewan Learning. (2006). Understanding the numbers.

9 Assessment for Learning is a Snapshot
Results from a large-scale assessment are a snapshot of student performance. The results are not definitive. They do not tell the whole story. They need to be considered along with other sources of information available at the school. The results are more reliable when larger numbers of students participate and when aggregated at the provincial and division level, and should be considered cautiously at the school level. Individual student mastery of learning is best determined through effective and ongoing classroom-based assessment. (Saskatchewan Ministry of Education, 2008)

10 Comparators: Types of Referencing
Criterion-referenced: Comparing how students perform relative to curriculum objectives, level attribution criteria (rubrics) and the level of difficulty inherent in the assessment tasks. If low percentages of students are succeeding with respect to specific criteria identified in rubrics, this may be an area for further investigation, and for planning intervention to improve student writing. (Detailed rubrics, OTL rubrics and test items can be sourced at Standards-referenced: Comparing how students performed relative to a set of professionally or socially constructed standards. Results can be compared to these standards to help identify key areas for investigation and intervention. (The standards set by the panel.) Refer to p. 2 in the detailed reports for more information

11 Comparators: Types of Referencing
Experience or self–referenced: Comparing how students perform relative to the assessment data gathered by teachers during the school year. Where discrepancies occur, further investigation or intervention might be considered. It is recommended that several sources of data be considered in planning. (E.g.. Comparing these results to current school data.) Norm-referenced: Comparing how students in a school performed relative to the performance of students in the division, region or project. Note cautions around small groups of students. Norm-reference comparisons contribute very little to determining how to use the assessment information to make improvements. (E.g.. Tables comparing the school, division and province.) Refer to p. 2 in the detailed reports for more information

12 Comparators: Types of Referencing
Longitudinal-referenced: Comparing how students perform relative to earlier years’ performance of students. Viewed across several years, assessment results and other evidence can identify trends and improvements. Four years of data for Math are available. Two years of data for Reading are available. It is important to note that identification of trends be done with extreme caution until more assessment cycles have been completed. Refer to p. 2 in the detailed reports for more information

13 Opportunity-to-Learn Elements as Reported by Students
MATH Preparation for and commitment to learn Persistence in solving math problems General Support for learning Support for learning in math READING Preparation for and commitment to learn Knowledge and use of reading strategies Home support for reading Look at the detailed Opportunity-to-Learn rubric. Discuss with your table group the key features of each criteria.

14 Opportunity-to-Learn Elements as Reported by Teachers
Availability and Use of Resources Classroom Instruction and Learning Math only: Approaches to Problem Solving Look at the detailed Opportunity-to-Learn rubric. Discuss with your table group the key features of each criteria.

15 Student Performance Outcome Results
Reading Comprehension Skills Explicit Implicit Critical By Strategy: Using Cueing Systems Connecting to Prior Knowledge Making Inferences/Predictions Noting Key Ideas and Finding Support Summarizing/Recalling/ Organizing Information Recognizing Author’s Message and Craft Reader Response

16 Student Performance Outcome Results
Math Content Skills 5 & 8 Number Patterns and Relationships Shape and Space Statistics and Probability Short Answer Math Challenge – integrated applications Estimation skills Computation (no calculator) Math 20 Content Skills Irrational Numbers Consumer Mathematics Polynomials and Rational Functions Quadratic Functions Quadratic Equations Probability Angles and Polygons Circles Estimations Math Challenge

17 Standards To help make meaningful longitudinal comparisons three main processes are used. Assessment items will be developed for each assessment cycle using a consistent table of specifications. The assessment items will undergo field-testing - one purpose of which is intended to inform the comparability of the two assessments. A process for setting of standards for each of the assessment items, so that any differences in difficulty between two assessments are accounted for by varying standards for the two assessments.

18 Opportunity-to-Learn and Performance Standards
In order to establish Opportunity-to-Learn and Performance standards for the 2009 Reading and Math Assessments, three panels were convened (one from each assessed grade), consisting of teachers from a variety of settings and post-secondary academics including Education faculty. The panelists studied each section of the 2009 assessments in significant detail and established expectations for both the performance elements and opportunity to learn.

19 Thresholds of Adequacy and Proficiency
Beginning Developing Adequate Proficient Insightful

20 Thresholds of Adequacy and Proficiency
Threshold of Adequacy Threshold of Proficiency 61% 14% Adequate Proficient & Beyond 61% of students achieved at least the adequate standard of performance for a particular measure 14% of students achieved at least the proficient standard of performance for a particular measure

21 Cut Scores On page 4 of the detailed reports you will find the cut scores. Cut scores indicate the percentage correct required for students to be classified as either excellent or sufficient with respect to OTL or proficient or adequate with respect to performance: Opportunity-to-Learn Elements Performance Component Excellent Standard Sufficient Standard Proficient Standard Adequate Standard 5 Level Scale Math & Reading – reported as percentages Math Challenges and Reader Response – 5 Level Scale

22 Effective Data Use Making significant progress in improving student learning and closing achievement gaps is a moral responsibility and a real possibility in a relatively short amount of time – two to five years. It is not children’s poverty or race or ethnic background that stands in the way of achievement. It is school practices and policies and the beliefs that underlie them that pose the biggest obstacles. Love, N., Stiles, K., Mundry, S., & DiRanna, K. (2008). Passion and principle ground effective data use. Journal of Staff Development, 29(4),

23 Mining the Report: Multiple Choice Questions
MATH Resource: MC answer key with reasons for distractors Grade 5 & 8 Table Grade.5 Grade 11 READING Resource: Explanation of reading strategies and levels of comprehension Table Grade.6 What patterns are you noticing? Identify strengths and weaknesses. Write them on sticky notes – one per note! Do any of these patterns extend to sub-groups? (Boys, Girls, First Nations & Métis) Refer to the Gender Comparisons.

24 Mining the Report: Math: Challenges, Computation & Estimation Reading: Reader Response
Integrated Applications Gr. 5 & 8 - Section 4 Gr. 11 – Section 3 Computation and Estimation Gr. 5 & 8 - Section 5 Gr. 11 – Section 4 READING Resource: Informational and Literary Text Coding Guides Table Grade.8 What patterns are you noticing? Identify strengths and weaknesses. Write them on sticky notes – one per note! Do any of these patterns extend to sub-groups? (Boys, Girls, First Nations & Métis) Refer to the Gender Comparisons.

25 Looking for Trends Refer to pages of the detailed report for Mathematics. What trends seem to be emerging? Write these on sticky notes as well. For reading, refer to pages 10-19, there will only be one other year of data to review. While this is not enough data to establish a trend, in what ways have results changed in comparison to the 2007 assessment? Write these on sticky notes as well.

26 Reflection on Findings pg. 9
In what ways does your school data in these areas confirm or contradict what you have found? For the school? For subpopulations? What other information might you require?

27 Student Performance Outcome Results
Take the sticky notes and classify them by the categories within the AFL report: Reading Comprehension Skills Explicit Implicit Critical By Strategy: Using Cueing Systems Connecting to Prior Knowledge Making Inferences/Predictions Noting Key Ideas and Finding Support Summarizing/Recalling/ Organizing Information Recognizing Author’s Message and Craft Reader Response

28 Student Performance Outcome Results
Take the sticky notes and classify them by the categories within the AFL report: Math Content Skills 5 & 8 Number Patterns and Relationships Shape and Space Statistics and Probability Short Answer Math Challenge – integrated applications Estimation skills Computation (no calculator) Math 20 Content Skills Irrational Numbers Consumer Mathematics Polynomials and Rational Functions Quadratic Functions Quadratic Equations Probability Angles and Polygons Circles Estimations Math Challenge

29 Lunch We resume at 12:45

30 Cause and Effect Analysis pp. 10-11
Choose one strength and one weakness to analyze. Using the supplied template, analyze the causal factors that contributed to that strength or weakness in the areas of: Instruction (Math Grade.2d; Reading Grade.13, .14) School Based Data/Student Work Demographics (pp ) Student Perceptions (Tables Grade.2a, 2b, 2c; Reading Grade.9, 10, 11, 12)

31 Designing Interventions
Using data itself does not improve teaching. Improved teaching comes about when teachers implement sound teaching practices grounded in cultural proficiency – understanding and respect for their students’ cultures – and a thorough understanding of the subject matter and how to teach it, including understanding student thinking and ways of making content accessible to all students. Love, N., Stiles, K., Mundry, S., & DiRanna, K. (2008). Passion and principle ground effective data use. Journal of Staff Development, 29(4),

32 Look for Teaching Strategies pg. 12
Choose one of the categories that you have identified. Place the name of that category in the middle of the bubble chart provided. Using the materials provided, begin a preliminary investigation of teaching strategies that are available. Discussion Questions Which strategies are well integrated in our practice? Which strategies might we study further and begin to integrate into our practice? What does the research say about how students work with this content?

33 Four Tasks of Action Planning – pg. 13
Decide on strategies for improvement. Agree on what your plan will look like in classrooms. Put the plan down on paper. Plan how you will know if the plan is working. Boudette, K., City, E. A., & Murnane, R. J. (2005). Data wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, MA: Harvard Education Press.

34 Writing Out The Plan pg.14 Using the supplied Implementation Indicators template, begin to draft the details of the plan as you work to achieving your goal. The supplied template is only a suggestion – you may create your own or use another of your own design. Boudette, K., City, E. A., & Murnane, R. J. (2005). Data wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, MA: Harvard Education Press.

35 EVIDENCE OF IMPLEMENTATION INDICATORS
Strategy: Estimation & Computations What we will see and hear in classrooms Evidence Collected by Teachers Teachers will teach strategies and processes for using estimation in computations. Teachers will model and encourage students to demonstrate more than one strategy for computations. Teachers will ask a variety of students to explain their reasoning. Students Students will be experimenting with a variety of ways to mentally solve numerical problems. Students will “talk through” their computation of a math problem. Students will be explaining how they reason. Classrooms Noise, as students collaborate and “talk through” their problems. Manipulatives, drawings, small groups. Processes or steps for mental computations. Student Work Students can clearly articulate processes for mental computations. Physical and graphic representations of solutions to problems. Increasing speed and fluency in computation skills. Boudette, K., City, E. A., & Murnane, R. J. (2005). Data wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, MA: Harvard Education Press. 35

36 EVIDENCE OF IMPLEMENTATION INDICATORS
Strategy: Making Inferences/Predictions What we will see and hear in classrooms Evidence Collected by Teachers Teachers will teach strategies for making inferences and predictions. Teachers will be seen using supports such as anticipation-reaction guides. Teachers will encourage students to make inferences and predictions in the materials they are listening to, reading, viewing. Students Students will be using a variety of graphic organizers and other supports such as anticipation-reaction guides. Students will articulate inferences and predictions with reasons. Students will evaluate the accuracy of their inferences and predictions. Classrooms Noise, as students collaborate. Samples of graphic organizers. Information on making inferences and predictions. Student Work Students will journal inferences and predictions with supporting reasons. Students will reflect on their inferences and predictions in writing. Boudette, K., City, E. A., & Murnane, R. J. (2005). Data wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, MA: Harvard Education Press. 36

37 Plan How You Will Know if the Plan is Working
Before implementing your plan, it is important to determine what type of data you will need to collect in order to understand whether students are moving towards the goal. Boudette, K., City, E. A., & Murnane, R. J. (2005). Data wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, MA: Harvard Education Press.

38 Put the Plan Down on Paper pg. 16
By documenting team members’ roles and responsibilities and specifying the concrete steps that need to occur, you build internal accountability for making the plan work. Identifying the professional development time and instruction your team will need and including it in your action plan lets teachers know they will be supported through the process of instructional improvement. Boudette, K., City, E. A., & Murnane, R. J. (2005). Data wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, MA: Harvard Education Press.

39 Advancing Assessment Literacy Modules – pg. 18
17 Modules designed to facilitate conversations and work with data for improvement of instruction. Publications Advancing Assessment Literacy Modules Download a PDF of a PowerPoint and accompanying Lesson Plan for use by education professionals in schools.

40 Key Word Cycle (Adapted from Anne Davies, 2009)
As you think about your learning today, create a key word cycle.

41 Outcomes Bringing context and meaning to the math and reading assessment project results; Initiating reflection and discussion among school staff members related to the math and reading assessment results; Encouraging school personnel to judiciously review and utilize different comparators (criterion-referenced, experience-referenced, standards-referenced, longitudinal-referenced, and normative-referenced) when judging assessment results; Modeling processes that can be used at the school level for building understanding of the links between assessment, curriculum and instruction among school staff and the broader community; and, Providing opportunity to discuss and plan at the school level


Download ppt "Understanding and Using Data to Plan for School Improvement"

Similar presentations


Ads by Google