1. Early Childhood Tracking Systems Hands On Workshop ESO Network 14 (Jackie Young) Prepared by Deena Abu-Lughod, SAF November 29, 2007 Participants: 4, 7, 20, 24, 59, 81, 88, 197, 209, 230, 280

2. Agenda Purpose Purpose Rationale from SQR Rationale from SQR Current Status Current Status Data Availability (DIBELS, ECLAS 2, ORA, CBM) Data Availability (DIBELS, ECLAS 2, ORA, CBM) Ideal System Criteria Ideal System Criteria Building a system using RESI Building a system using RESI Evaluation Evaluation

3. Purpose Take Writing Prompt 1 Take Writing Prompt 1 For 30 seconds, think about the prompt. For 30 seconds, think about the prompt. For 90 seconds, write down as much as you can about what you believe the purpose of today should be. Do not worry about spelling or punctuation or grammar. For 90 seconds, write down as much as you can about what you believe the purpose of today should be. Do not worry about spelling or punctuation or grammar. Place your paper on the wall. Place your paper on the wall. A volunteer will read all submissions. A volunteer will read all submissions. What commonalities do you see? What commonalities do you see? As a group, formulate a statement that represents the consensus. As a group, formulate a statement that represents the consensus.

4. Rationale: Expectations from SQR Quality Statement 1 – Gather Data: School leaders and faculty consistently gather and generate data and use it to understand what each student knows and is able to do, and monitor the students progress over time. To what extent do school leaders and faculty gather, generate, and utilize data 1.1 to provide an objective, constantly updated understanding of the performance and progress of each student, classroom and grade level in each subject area 1.2 to provide an objective, constantly updated understanding of the performance and progress of special education students 1.3 to provide an objective, constantly updated understanding of the performance and progress of English language learners 1.4 to provide an objective, constantly updated understanding of the performance and progress of ethnic groups, boys and girls and all other categories of interest to the school 1.5 to compare the schools current and past performance and progress, and compare performance and progress of different students, classrooms and grades, and in different subject areas 1.6 to compare the schools performance and progress to that of similar schools 1.7 with sufficient training, management systems and structures to enable data to be used effectively to inform planning and instruction and to track the progress of students over time

5. Rationale: Expectations from SQR Quality Statement 5 – Monitor and Revise: The school has structures for monitoring and evaluating each students progress throughout the year and for flexibly adapting plans and practices to meet its goals for accelerating learning. To what extent do... Δ 5.1 the schools plans for improving student outcomes include interim goals that are objectively measurable and have suitable time frames for measuring success and adjusting 5.2 the schools plans for improving teacher outcomes include interim goals that are objectively measurable and have suitable time frames for measuring success and adjusting 5.3 faculty use periodic assessments and other diagnostic tools to measure the effectiveness of plans and interventions for individual students and groups of students in key areas 5.4 faculty use the information generated by periodic assessments and other progress measures and comparisons to revise plans immediately in order to reach stated goals 5.5 school leaders track the outcomes of periodic assessments and other diagnostic measures and use the results to makes strategic decisions to modify practices to improve student outcomes 5.6 school leaders and teachers use each plans interim and final outcomes to drive the next stage of goal setting and improvement planning 5.7 the principal and school community have a clear vision for the future development of the school and implement procedures and systems to effect change

6. Current Status: Monitoring performance and progress Read the following Indicators and think about how your school currently meets the expectations of the Quality Review: Read the following Indicators and think about how your school currently meets the expectations of the Quality Review: 1.4: 1.4: to provide an objective, constantly updated understanding of the performance and progress of ethnic groups, boys and girls and all other categories of interest to the school 5.5: 5.5: school leaders track the outcomes of periodic assessments and other diagnostic measures and use the results to make strategic decisions to modify practices to improve student outcomes

7. Current Status: Monitoring performance and progress Take Writing Prompt #2. At the signal, think about the prompt for 30 seconds. At the signal, think about the prompt for 30 seconds. For 90 seconds, write as much as you can as a response. Do not worry about spelling, punctuation or grammar. For 90 seconds, write as much as you can as a response. Do not worry about spelling, punctuation or grammar. Share your ideas with your partner. (2 minutes) Share your ideas with your partner. (2 minutes) Reverse roles. (2 minutes) Reverse roles. (2 minutes) As a pair, identify 1 key idea for each indicator (or 2 ideas if you both wrote on the same indicator) and report out to the group. As a pair, identify 1 key idea for each indicator (or 2 ideas if you both wrote on the same indicator) and report out to the group.

8. Our Beliefs: How do children learn to write? Take Writing Prompt #3 Take Writing Prompt #3 For 30 seconds, think about the prompt. For 30 seconds, think about the prompt. For 90 seconds, write down as much as you can about how you believe children learn to write. Do not worry about spelling or punctuation or grammar. For 90 seconds, write down as much as you can about how you believe children learn to write. Do not worry about spelling or punctuation or grammar.

9. Our Beliefs Text Based Discussion based on Whats Next for this Beginning Writer? Text Based Discussion based on Whats Next for this Beginning Writer?

10. Data Availability: DIBELS Provides overall instructional recommendation and scores on sub- components (ISF, LNF, PSF, NWF, ORF, WUF) Provides overall instructional recommendation and scores on sub- components (ISF, LNF, PSF, NWF, ORF, WUF) Available as pdf and is exportable as a spreadsheet Available as pdf and is exportable as a spreadsheet

11. Data Availability: ECLAS 2 Online access (delayed) Online access (delayed) Excel spreadsheet (through network) Excel spreadsheet (through network) High correlation between EOY Grade 3 Sight Words score and Grade 3 ELA High correlation between EOY Grade 3 Sight Words score and Grade 3 ELA Indicates mastery levels for various subtests Indicates mastery levels for various subtests

12. Data Availability: ORA Administered and scored by hand Administered and scored by hand No electronic tracking system developed No electronic tracking system developed

13. Data Availability: DRA/RIGBY Administered and scored by hand Administered and scored by hand Tracking system under development Tracking system under development

14. Data Availability: EDM Tracking Developed by Teach for America Developed by Teach for America Each question on each unit test is coded by performance indicator and standard Each question on each unit test is coded by performance indicator and standard Teacher enters data (1 for correct; 0 for incorrect) for each student and each item. Teacher enters data (1 for correct; 0 for incorrect) for each student and each item. Spreadsheet automatically computes student and classroom average. Spreadsheet automatically computes student and classroom average. Format allows for easy detection of patterns by indicator Format allows for easy detection of patterns by indicator Requires some Excel skills Requires some Excel skills

15. Data Availability: Other 1

16. Data Availability: Other 2

17. Constructing Criteria Form groups based on Sets 1, 2 or 3. Form groups based on Sets 1, 2 or 3. What elements do you like about the tracking system? What elements do you like about the tracking system? What elements are troubling? What elements are troubling? What changes would you make? What changes would you make? What would a complete system look like? What would a complete system look like?

18. Elements of a good system

19. Create your own system On the planning worksheet, identify the data elements you would like to include in your system. On the planning worksheet, identify the data elements you would like to include in your system. Consider the order in which you would like the elements to appear. Consider the order in which you would like the elements to appear. When you are ready, load up the RESI report on your computer. Make a copy of it (so you can save the original in case you make a mistake). When you are ready, load up the RESI report on your computer. Make a copy of it (so you can save the original in case you make a mistake).

20. Our own Writing Prompts

21. Evaluation Review purpose Review purpose What did we achieve? What did we achieve? Do we need to continue? Do we need to continue? Complete PMI (Plus, Minus, Interesting) Complete PMI (Plus, Minus, Interesting)

22. Progress Monitoring Progress monitoring (PM) is conducted frequently and is designed to: Progress monitoring (PM) is conducted frequently and is designed to: Estimate rates of student improvementEstimate rates of student improvement Identify students who are not demonstrating adequate progressIdentify students who are not demonstrating adequate progress Compare the efficacy of different forms of instruction and design more effective, individualized instructional programs for problem learnersCompare the efficacy of different forms of instruction and design more effective, individualized instructional programs for problem learners

23. What Is the Difference Between Traditional Assessments and PM? Traditional assessments: Traditional assessments: LengthyLengthy Administered irregularlyAdministered irregularly Feedback is delayedFeedback is delayed May lack meaningful contexts (eg. scores may be based on national norms and a teachers classroom may differ from the national student sample)May lack meaningful contexts (eg. scores may be based on national norms and a teachers classroom may differ from the national student sample)

24. What Is the Difference Between Traditional Assessments and CBM? Curriculum-Based Measurement (CBM) is one way to monitor progress: Curriculum-Based Measurement (CBM) is one way to monitor progress: CBM provides an easy and quick method to measure student progressCBM provides an easy and quick method to measure student progress Teachers can analyze student scores and adjust student goals and instructional programsTeachers can analyze student scores and adjust student goals and instructional programs Student data can be compared to by classroom or subgroupStudent data can be compared to by classroom or subgroup

25. Curriculum-Based Assessment Features: Features: Materials align with school curriculumMaterials align with school curriculum Measurement is frequentMeasurement is frequent Assessment information can be used to formulate instructional decisionsAssessment information can be used to formulate instructional decisions

26. Basics of CBM Monitors student progress across entire school year Monitors student progress across entire school year Meaningful, accurate results can help quantify short- and long-term student gains Meaningful, accurate results can help quantify short- and long-term student gains Probes for reading, spelling, writing and math are brief and easy to administer Probes for reading, spelling, writing and math are brief and easy to administer Teachers quickly determine whether an intervention is helping a student Teachers quickly determine whether an intervention is helping a student

27. Steps for Conducting CBM Step 1: Place Students in a Task for Progress Monitoring Step 1: Place Students in a Task for Progress Monitoring Step 2: Identify the Level for Material for Monitoring Progress Step 2: Identify the Level for Material for Monitoring Progress Step 3: Administer and Score Probes Step 3: Administer and Score Probes Step 4: Graph Scores Step 4: Graph Scores Step 5: Set Goals Step 5: Set Goals Step 6: Apply Decision Rules to Graphed Scores to Know When to Revise Programs and Increase Goals Step 6: Apply Decision Rules to Graphed Scores to Know When to Revise Programs and Increase Goals Step 7: Use the Data Qualitatively to Describe Students Strengths and Weaknesses Step 7: Use the Data Qualitatively to Describe Students Strengths and Weaknesses

28. Step 1: Place Students in a Written Expression Task for Progress Monitoring Decide which task or grade level of material is developmentally appropriate Decide which task or grade level of material is developmentally appropriate Monitor writing fluency by using Written Expression CBM in Grades 1– 12 (as soon as students can write sentences) Monitor writing fluency by using Written Expression CBM in Grades 1– 12 (as soon as students can write sentences)

29. Step 2: Identify the Level of Material for Monitoring Progress For Written Expression: For Written Expression: Prompts should be simple in sentence structure and tap age-appropriate background knowledge of studentsPrompts should be simple in sentence structure and tap age-appropriate background knowledge of students Prompts should represent experiences of a wide range of school-age studentsPrompts should represent experiences of a wide range of school-age students

30. Step 3: Administer and Score Written Expression CBM Administer to entire class at one time Administer to entire class at one time Present students with a story starter: Present students with a story starter: Use a familiar themeUse a familiar theme End the starter mid-sentenceEnd the starter mid-sentence Write the starter at top of student CBM probeWrite the starter at top of student CBM probe Give students a set time to write Give students a set time to write Score Written Expression CBM probes Score Written Expression CBM probes

31. Sample Probes I was on my way home from school and … I was on my way home from school and … I was talking to my friends when all of a sudden … I was talking to my friends when all of a sudden … It was a dark and stormy night … It was a dark and stormy night … One day I found the most interesting thing … One day I found the most interesting thing … One night I had a strange dream about … One night I had a strange dream about … I found a note under my pillow that said … I found a note under my pillow that said … The cave was very dark and … The cave was very dark and … One day I went to school but nobody was there except me … One day I went to school but nobody was there except me …

32. Step 3: Administer and Score Written Expression CBM Read the story starter aloud to the classRead the story starter aloud to the class Give students 30 seconds to think about their writingGive students 30 seconds to think about their writing Clarify whether students can draw or write notes Clarify whether students can draw or write notes Re-read story starter aloud to the classRe-read story starter aloud to the class Students write for 3 minutes (early childhood)Students write for 3 minutes (early childhood)

33. Step 3: Scoring Written Expression Several ways to score: Several ways to score: Total words writtenTotal words written Total words spelled correctlyTotal words spelled correctly Total words written minus incorrectly spelled wordsTotal words written minus incorrectly spelled words Correct word sequences (CWS)Correct word sequences (CWS) Correct minus Incorrect Word Sequences (CIWS)Correct minus Incorrect Word Sequences (CIWS) Keep the scoring system consistent throughout the entire school yearKeep the scoring system consistent throughout the entire school year

34. Step 3: Administer and Score Written Expression CBM Words written: Words written: Calculate the total number of words writtenCalculate the total number of words written Correct spelling, word usage, capitalization, and punctuation are ignoredCorrect spelling, word usage, capitalization, and punctuation are ignored

35. Step 3: Administer and Score Written Expression CBM The cave was very dark and I try to close my eyes, so I couldnt see anything, but that didnt help. Than I hear some one breathing. I try to stream, but nother came out. The breathing became close and close to me, and the worst Part was that I couldnt see athing. At first I thought meslef that I an Just emaging stuff.

36. Step 3: Administer and Score Written Expression CBM I was on my way home from school and I saw my friend and we rund bilk to my hous. Bot she had to go to the bath room at the palk. So we played a little bit and we whant to the store and bot some goodes.

37. Step 3: How to Administer and Score Written Expression CBM I was on my way home from school and I saw my friend and we rund bilk to my hous. Bot she had to go to the bath room at the palk. So we played a little bit and we whant to the store and bot some goodes.

38. Step 3: Administer and Score Written Expression CBM Words spelled correctly: Words spelled correctly: Calculate the total number of words spelled correctlyCalculate the total number of words spelled correctly Any correctly spelled English word is counted as correctAny correctly spelled English word is counted as correct Proper usage, capitalization, and punctuation are ignoredProper usage, capitalization, and punctuation are ignored

39. Step 3: How to Administer and Score Written Expression CBM The cave was very dark and I try to close my eyes, so I couldnt see anything, but that didnt help. Than I hear some one breathing. I try to stream, but nother came out. The breathing became close and close to me, and the worst Part was that I couldnt see athing. At first I thought meslef that I an Just emaging stuff.

40. Step 4: Graphing Scores Once CBM data has been collected, graph student scores Once CBM data has been collected, graph student scores Graphs provide teachers with a straightforward way of: Graphs provide teachers with a straightforward way of: Reviewing student progressReviewing student progress Monitoring appropriateness of student goalsMonitoring appropriateness of student goals Judging adequacy of student progressJudging adequacy of student progress Comparing and contrasting successful and unsuccessful instructional aspectsComparing and contrasting successful and unsuccessful instructional aspects

41. Step 4: Graphing Scores Teachers can make decisions about short- and long-term progress Teachers can make decisions about short- and long-term progress Graphs help teachers set ambitious, but realistic goals Graphs help teachers set ambitious, but realistic goals Graphs provide teachers with actual data to help revise and improve student instructional programs Graphs provide teachers with actual data to help revise and improve student instructional programs

42. Step 4: Graphing Scores Two options for creating CBM graphs: Two options for creating CBM graphs: Create own students graphs using graph paper and pencilCreate own students graphs using graph paper and pencil Create graphs using computer software (such as Microsoft Excel)Create graphs using computer software (such as Microsoft Excel) Create a master graph Create a master graph Vertical axis: range of scores of all students in the classVertical axis: range of scores of all students in the class Horizontal axis: number of weeks of instructionHorizontal axis: number of weeks of instruction Master graph serves as a template Create a graph for each individual student for interpretation Master graph serves as a template Create a graph for each individual student for interpretation

43. Step 4: Graphing Scores

44. Step 5: Setting Goals End-of-year performance goals End-of-year performance goals Three options: Three options: End-of-year benchmarkingEnd-of-year benchmarking Intra-individual frameworkIntra-individual framework NormsNorms

45. Step 5: Option #1 Option #1: End-of- year benchmarking Option #1: End-of- year benchmarking Identify end-of-year benchmarkIdentify end-of-year benchmark Mark as an X on the graph at the date marking the end of the yearMark as an X on the graph at the date marking the end of the year Goal-line is drawn between median of first three CBM scores and end-of-year performance goalGoal-line is drawn between median of first three CBM scores and end-of-year performance goal GradeBenchmark 1st Written Expression Use school or other norms 2nd Written Expression Use school or other norms 3rd Written Expression Use school or other norms

46. Step 7: Use the CBM Database Qualitatively to Describe Student Strengths and Weaknesses Written Expression CBM Written Expression CBM Story-IdeaOrganization-CohesionConventions-Mechanics 4 -includes characters, but they are not original, often coming from movies -delineates a plot, although it is not as clear as 5 -contains some original ideas but is it fairly predictable -contains some detail -includes descriptors (adverbs and adjectives) -words choice: contains some descriptors (adverbs and adjectives) and some colorful, infrequently used, and/or long words 4 -story has somewhat of a beginning, middle, and an end -events appear somewhat random, but some organization exists -sample may contain some transitions to help with organization (finally, then, next, etc.) -story often contains too many events, disrupting cohesion 4 -sentence structure generally is accurate but not as good as 5 -spelling does not hinder readability too much -sometimes contains dialogue -handwriting is legible -punctuation does not effect readability too much -word usage generally is correct (s,v,o/homophone/s-v agreement)

47. Step 7: Use the CBM Database Qualitatively to Describe Student Strengths and Weaknesses Written Expression CBM Written Expression CBM Story-IdeaOrganization-CohesionConventions-Mechanics 3 -characters are predictable and undeveloped -plot is somewhat haphazard -may or may not contain original ideas -lacks detail -word choice is somewhat predictable only sometimes contains descriptors (adverbs and adjectives) 3 -somewhat of a plot exists but story may still lack a beginning, middle, or an end -events are somewhat random -often lacks transitions -sometimes lacks referents 3 -sentence structure has a few problems -spelling is somewhat of a problem -may use dialogue but does not punctuate it correctly -handwriting is legible -punctuation is fair -problems sometimes occur with word usage (s,v,o/homophone/s-v agreement)

48. Step 7: Use the CBM Database Qualitatively to Describe Student Strengths and Weaknesses Written Expression CBM Written Expression CBM Story-IdeaOrganization-CohesionConventions-Mechanics 2 -includes few if any characters -plot is not developed or apparent -contains virtually no original ideas -detail is significantly absent -events are very predictable -word choice is predictable, lacking descriptors (adverbs and adjectives) 2 -plot lacks organization into a beginning, middle and an end -events are random, lacking in cohesion -lacks transitions -often lacks referents 2 -sentence structure makes story difficult to read -spelling makes it difficult to read -may use dialogue but does not punctuate it correctly -handwriting is not very legible -punctuation is inconsistent and problematic -word usage is problematic (s,v,o/homophone/s-v agreement)