Data Analysis Protocols: An Overview A “Ground Up Approach” to Effective Data Use 1

Why Use Assessment Data Quality School Review (QSR) Rubric: Indicator 1.6: The principal ensures that classroom level instruction is adjusted based on formative and summative results from aligned assessments. Indicator 3.3: Teachers use frequent checks for understanding throughout each lesson to gauge student learning, and to inform, monitor and adjust instruction. Indicator 3.5: Teachers demonstrate the necessary skills to use multiple measures of data, including the use of diagnostic, formative and summative assessment data, to differentiate instruction to improve student achievement. Indicator 4.2: Teachers and school leaders collect classroom level data to verify that the adopted and aligned CCSS curriculum is the “taught” curriculum. What is the Quality School Review (QSR)? The QSR is a baseline needs assessment of Priority and Focus Schools with school quality indicators aligned to the eight turnaround principles. The QSR replaces the CAPA Review. Replace/modify slide as needed. 2

Why Use Assessment Data Why are teachers learning this? Danielson Framework for Teaching: Domain 1e: Designing coherent instruction. Domain 1f: Designing student assessments. Domain 3d: Using assessment in instruction. Domain 4a: Reflecting on teaching. Domain 4b: Maintaining accurate records. Though this reflects instructional components for districts/schools using the Danielson Framework, most other teacher evaluation models will have similar components. Replace as needed. 3

Why Use Assessment Data Here’s some supporting research: Abbott, D. V. (2008). A functionality framework for educational organizations: Achieving accountability at scale. In E. Mandinach & M. Honey (Eds.), Data driven school improvement: Linking data and learning (pp. 257–276). New York: Teachers College Press. Brunner, C., Fasca, C., Heinze, J., Honey, M., Light, D., Mandinach, E., et al. (2005). Linking data and learning: The Grow Network study. Journal of Education for Students Placed At Risk, 10(3), 241–267. Halverson, R., & Thomas, C. N. (2007). The roles and practices of student services staff as data-driven instructional leaders. In M. Mangin & S. Stoelinga (Eds.), Instructional teachers leadership roles: Using research to inform and reform (pp. 163–200). New York: Teachers College Press. Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education, 112(4), 496–520 Liddle, K. (2000). Data-driven success: How one elementary school mined assessment data to improve instruction. American School Board Journal. Retrieved April 19, 2009, from http://www.asbj. Mandinach, E. B., Honey, M., Light, D., Heinze, C., & Rivas, L. (2005, June). Creating an evaluation framework for data-driven decision-making. Paper presented at the National Educational Computing Conference, Philadelphia, PA. 4

Part One: Daily Data Use A “Ground Up Approach” to Daily Data Use 5

An Effective Data-Use Model Why do we so often stop here? Source: http://ies.ed.gov/ncee and http://ies.ed.gov/ncee/wwc/publications/practiceguides/ 6

Increased Student Learning Daily Data101 Backward Planning Increased Student Learning 1) Quarterly/ Benchmark Assessments should lead to… SHORT, aligned assessments. 2) SHORT, assessments aligned to Benchmarks should lead to… Creating aligned end-of-class DOLs. 3) Creating aligned end-of-class DOLs should lead to… Responding to daily DOL data in real time. 4) Responding to daily DOL data in real time should lead to… Quick, targeted remediation. 5) Quick, targeted remediation should lead to… BETTER STUDENT OUTCOMES DOL: Demonstration of learning. It can also be considered a CFU (check for understanding). Though there are differences between the two, the basic concept is that if students are provided with new concepts or understandings, then a teacher should assess the level of student mastery of the new information.

Getting to a Daily Data-Use Mindset Data analysis is really a daily, minute-to-minute process. The above graphic shows one method for imagining the process in action.

When to Intervene: A Suggestion Demonstration of Learning (DOL) at the end of lesson Daily What to do next based on your results Metric Mastery of 80% - 99% Teacher works with a small group so they can catch up. Mastery of 60% - 79% Teacher uses as next-day’s Do Now (e.g. error analysis or a peer review) and does a mini-review. Mastery of 0% - 60% Teacher works with small groups AND puts independent Learning Activity related to the skill in the classroom for students’ in-class use. Ignoring Data = Ignoring the Learner’s Needs

Part Two: Data Analysis Protocols Frameworks for Conversations about Results

Pre-Assessment Resources Teacher Tools Pre-Assessment Resources

Prior to Starting a New Unit edConnect assessments are available to preview in advance of the testing window Use this as an opportunity to reflect and plan strategies for preparing students without “teaching to the test” or, worse, teaching the actual test.

Teacher Assessment Analysis Tools Consider using one of the following tools to reflect on the End-of-Unit assessments to aide in unit planning. Option 1 EUA Pre-Assessment Reflection Form Option 2 Assessment Analysis Tool

Post-Assessment Resources Teacher Tools Post-Assessment Resources

Teacher Reflection Protocol Post-EUA Administration Reflection Questions: What trends do you find in the data? To what would you attribute the results? What questions come to mind when you review the data? What recommendations would you make to improve student performance? The reflection questions above provide the teacher with an entry point for assessment data analysis. Data Analysis Protocols typically ask the same or similar questions. The end goal of the reflection process is to determine the nexts steps necessary to help students become proficient in the concepts being assessed.

Some Sample Teacher DAPs Friendly Guide for Making Sense of Data Link This is a basic form teachers can use to reflect on student performance. Data Analysis Planning Form This is another form teachers can use to reflect on student performance. edConnect Assessment Reflection form This form was developed for use in a school using edConnect during 2014-2015 SY. Click the links above to view some examples.

Assessment-Reflection Resources Student Tools Assessment-Reflection Resources

Teach Students to Examine Their Own Data Before test, have students set goals [See Hattie’s “Influences on Student Achievement”] See this sample student survey After test, have students complete a self-reflection: Student Self-Reflection Form Sample Also be sure teachers do the following: Explain student expectations and the assessment criteria. Provide feedback to students that is timely, specific, well-formatted, and constructive. Provide tools that help students learn from feedback. Use students’ data analyses to guide instructional changes.

Teach Students to Examine Their Own Data Teachers can easily collect students’ self-reflection data: Student Feedback Form Sample Form Results

Part Three: Next Steps Monitor and Adjust

Principal as Instructional Leader QSR Indicator 1.6 Principal as Instructional Leader Indicator 1.6: The principal ensures that classroom level instruction is adjusted based on formative and summative results from aligned assessments. A data management system provides teachers with analytic tools to gain insight into how students are performing and how to design ongoing instruction. Students who are not mastering lesson objectives are quickly identified and provided additional instructional supports until they achieve mastery.

Ingredients Needed Capacity to design and implement frequent, aligned assessments (ideally on a weekly basis). Capacity to quickly and easily access student performance data. Ability to meet weekly to discuss student performance citing REAL data from assessments. Capacity to monitor student learning during and immediately following instruction.

Need help? What do you do when you need assistance? Bruce Henecker Acting Director of Data, Assessment, and Planning New Jersey Department of Education bruce.henecker@doe.state.nj.us Replace contact information with a local contact as needed

In closing...

Effective Data Use Data Trend Analysis... is a daily practice informs interventions informs lesson planning INCREASES STUDENT LEARNING You’ll know this is happening when… Reflection forms are completed “Problem of practice” is addressed Gradebook reflects student achievement