1. 10705ZZ4211 Pennsylvania Benchmarks Data Analysis and Discovery (Root Cause Analysis)

2. © 2006 Success for All Foundation2 Component 1: Data Analysis and Discovery

3. © 2006 Success for All Foundation3 Data Dialogues PRIMARY QUESTIONS How many proficient? SECONDARY QUESTIONS Strengths/Weakness TERTIARY QUESTIONS Why/Discovery/Root Causes

4. © 2006 Success for All Foundation4 Primary Questions Primary questions are those that closely match your most essential goals. They tend to be very, very simple.

5. © 2006 Success for All Foundation5 Analyzing Proficiency Reports Primary Questions How many students are proficient in reading/math? How many students in this subgroup are proficient? How many students do you need in that subgroup to make AYP? How many more quarters until the PSSA is given? Who are the students just below proficiency? How many points do these students need to move up to be proficient? How many points per quarter?

6. © 2006 Success for All Foundation6 Member Center Primary Question Tools Proficiency Reports (Under Testing Center – Review/Enter Scores) RP 3508: Predicted State Results (Under Reports – School Reports) RP 3299: Student Data Sheets (Under Reports – School Reports) RP 3434: 4Sight Proficiency Projections (Must have entered last year’s AYP percentages.)

7. © 2006 Success for All Foundation7 Planning For and Monitoring AYP From the Member Center Menu Select: Reports School Reports AYP Reports RP-3533 Measures RP-3379 Projections by Grade RP-3434 4Sight Proficient Projections Caution: These reports are only as accurate as the student data in the Member Center.

8. © 2006 Success for All Foundation8 Data Dialogues PRIMARY QUESTIONS How many proficient? SECONDARY QUESTIONS Strengths/Weakness TERTIARY QUESTIONS Why/Discovery/Root Causes

9. © 2006 Success for All Foundation9 Secondary Questions Secondary questions discover what strengths and weaknesses are according to student learning. They tend to be more specific.

10. © 2006 Success for All Foundation10 Analyzing Subscale Data Secondary Questions What are the strengths identified on the graph? What are the areas for concern? If the graph is flat, are the students proficient readers or close to nonreaders?

11. © 2006 Success for All Foundation11 Member Center Secondary Question Tools Subscale Reports (Under Testing Center – Review/Enter Scores) RP 3304: Subscale Test Results (Under Reports – School Reports) RP 3474: Individual Student Test Results (Under Reports – School Reports – Prints a report for each individual student)

12. © 2006 Success for All Foundation12 Data Dialogues PRIMARY QUESTIONS How many proficient? SECONDARY QUESTIONS Strengths/Weakness TERTIARY QUESTIONS Why/Discovery/Root Causes

13. © 2006 Success for All Foundation13 Tertiary Questions Tertiary questions ask why. They seek to discover root causes for the identified strengths or areas of concern.

14. © 2006 Success for All Foundation14 Tertiary Question Probes Why do our students not perform according to state standards? Why do our students not know what they should know? Why are the students unable to use what they know? Why are our subgroups performing below?

15. © 2006 Success for All Foundation15 Student Demographics Attendance Subgroups Transiency Suspensions Curriculum Alignment Balance Instruction and Preparation Alignment Cycle of Effective Instruction Quality of Program Implementation Classroom Management Professional Development School Processes Assessment and Regrouping Intervention Resources Tutoring Solutions Team School Programs Teacher Selection Scheduling Leadership Organizational Culture Beliefs Values External Factors Parent Involvement Health Systems Approach to Discovery/Root Causes Determine Root Causes Related to Targeted Area of Concern © 2006 Success for All Foundation15

16. © 2006 Success for All Foundation16 Attendance Subgroups Ethnicity Gender Language Special Education Transiency Suspensions Student Demographics

17. © 2006 Success for All Foundation17 Alignment Balance Rigor Curriculum

18. © 2006 Success for All Foundation18 Alignment Cycle of Effective Instruction Active Instruction: Teach, Model, Guide Practice Partner/Team Practice Individual Mastery and Accountability Quality of Program Implementation Classroom Management Professional Development Instruction and Preparation

19. © 2006 Success for All Foundation19 Assessment and Regrouping Intervention Resources Tutoring School Programs Teacher Selection Scheduling Leadership Principal Leadership Team Literacy/Math Coaches School Processes

20. © 2006 Success for All Foundation20 Beliefs Values Artifacts Common Language Celebrations Organizational Culture

21. © 2006 Success for All Foundation21 Parental Involvement Health Socioeconomic Status Shelter Nutrition Parent Educational Level Community Crime Rate External Factors

22. © 2006 Success for All Foundation22 Data Analysis Identify Proximity to Goal(s). Identify Areas of Concern Develop Target(s). Discovery Brainstorm Root Causes. Prioritize and verify Root Causes with data. Select the Root Causes that are most important and doable. Keep the number low. Solutions Design interventions to address identified Root Causes. Evaluate results and readjust. Steps for Data Analysis, Discovery, and Solution Identification

23. © 2006 Success for All Foundation23 Data Analysis Identify Proximity to Goal(s). Identify Areas of Concern Develop Target(s). Discovery Brainstorm Root Causes. Prioritize and verify Root Causes with data. Select the Root Causes that are most important and doable. Keep the number low. Solutions Design interventions to address identified Root Causes. Evaluate results and readjust.

24. © 2006 Success for All Foundation24 Member Center and 4Sight Discovery Tools RP 3491: 4Sight Item Analysis Report: (Under Reports – School Reports) 4Sight Pennsylvania Benchmark Reporting Categories: (Admin. Guide – PA standards condensed version) 4Sight Pennsylvania Benchmark Answer and Alignment Guide: (Admin. Guide – Reporting Category, PA Standard, Assessment Anchor)

25. © 2006 Success for All Foundation25

26. © 2006 Success for All Foundation26 © 2006 Success for All Foundation26 Root Cause Analysis: Clarification Clarification Basal uses primarily narrative text No explicit instruction in clarification at the word or paragraph level Inadequate time spent On instructing how to read expository text Lack of PD on clarification with expository text No consistent reading block. MS have classroom management issues No school wide measurable goals on clarification Teachers and tutors not aligned Leadership not monitoring Attendance low Homework return low Homework not aligned Parents unaware of school focus and child’s skill level

27. © 2006 Success for All Foundation27 Data Analysis Identify Proximity to Goal(s). Identify Areas of Concern Develop Target(s). Discovery Brainstorm Root Causes. Prioritize and verify Root Causes with data. Select the Root Causes that are most important and doable. Keep the number low. Solutions Design interventions to address identified Root Causes. Evaluate results and readjust. Steps for Data Analysis, Discovery, and Solution Identification

28. © 2006 Success for All Foundation28 Verifying Root Causes Consult additional records/data (such as attendance). Review curriculum guides. Analyze student assignments. Conduct classroom walk-throughs. Do a school artifact inventory. Review office referral records. Etc.

29. © 2006 Success for All Foundation29 Data Does Not Fix Problems “Disaggregating data is not a problem-solving strategy. Rather it’s a problem-finding strategy.” — Lezotte (1999)

30. © 2006 Success for All Foundation30 Data Analysis Identify Proximity to Goal(s). Identify Areas of Concern Develop Target(s). Discovery Brainstorm Root Causes. Prioritize and verify Root Causes with data. Select the Root Causes that are most important and doable. Keep the number low. Solutions Design interventions to address identified Root Causes. Evaluate results and readjust. Steps for Data Analysis, Discovery, and Solution Identification

31. © 2006 Success for All Foundation31 Sample Improvement Plan Data Analysis Instructional Planning Form Plans for to (dates) Student Group: All students Target/Objective: All students will use clarification strategies when reading independently narrative and expository text. Success Indicator: Students will score 90% or higher on basic comprehension when reading material at an independent level. Action StepsResourcesFacilitatorTimeline Initial Training on teaching clarification strategies. Trainer Teacher Guides Student Practice Materials & Videos Reading CoachAugust 15 Initial coaching on teaching clarification strategies. Trainer Clarification strategies use data Completed lesson plans. Reading CoachSept. 30

32. © 2006 Success for All Foundation32 Designing Improvement Plans (1 year or 1 quarter) Turn Primary Root Causes into Action Steps Improvement Plans Include: 1.Target(s) 2.Action Steps 3.Resources Needed 4.Facilitator/Point Person for Each Step 5.Timeline 6.Success Indicator(s)

33. © 2006 Success for All Foundation33 Component 2: Using 4Sight to Inform Professional Development

34. © 2006 Success for All Foundation34 Results-Based Professional Development Identifies desired improvement in student achievement. Identifies a plan to help teachers achieve these results with their students. Gives teachers an opportunity to gain new knowledge and behaviors.

35. © 2006 Success for All Foundation35 Traditional vs. Results-Based Professional Development TraditionalResults-Based OpportunitiesResponsibilities You Might Want to GoProfessional Responsibility Staff DesiresStudent Needs Focus: Good Things to Do Focus: Needs of Students Decision by FeelingsDecision by Data

36. © 2006 Success for All Foundation36 Traditional vs. Results-Based Professional Development TraditionalResults-Based Many Topics AddressedOne Topic in Depth Reactive to ChangeProactive to Change Primarily Workshops Workshops with Implementation Evaluated Based on Satisfaction Evaluated Based on Student Data Awareness, KnowledgeSkill Attainment Passive Participants Participants Partners in Process

37. © 2006 Success for All Foundation37 Key Components Knowledge/Theory Modeling Demonstration Low-Risk Practice Feedback On-the-Job Practice and Feedback

38. © 2006 Success for All Foundation38 Knowledge/Theory What: rationale and description of techniques and uses How: readings, lectures, videos, discussions Value: increase awareness and conceptual understanding Note: Used alone it rarely results in skill acquisition nor does it transfer to the classroom.

39. © 2006 Success for All Foundation39 Modeling/Demonstration What: enactment of strategy or skill How: simulation with students/participants Value: mastery of knowledge related to skill Note: Modeling alone seldom results in transfer to the classroom.

40. © 2006 Success for All Foundation40 Low-Risk Practice What: nonthreatening experience with new skill How: simulation with a small group Value: begin to acquire new skill/strategy Note: Low-risk practice is a way to develop competence.

41. © 2006 Success for All Foundation41 Feedback on Low-Risk Practice What: provides for reflection on implementation How: self-reflection or provided by peers/coaches Value: increased awareness of one’s skills Note: Low-risk practice changes behaviors over the long term.

42. © 2006 Success for All Foundation42 On-the-Job Practice What: self-analyze and make adjustments for improvement How: uses consultants, peers, supervisors, and trainers Value: builds support, collegiality, and commitment Note: On-the-job practice is necessary for internalization of learning.

43. © 2006 Success for All Foundation43 Professional-Development Formats External Consultants Internal Consultants Observation of Others Collaborative Groups Literature

44. © 2006 Success for All Foundation44 External Consultants Who: trainers/consultants from intervention and curriculum providers, department of education staff, university staff Plus: bring in outside knowledge and strategies Delta: not part of the day-to-day implementation

45. © 2006 Success for All Foundation45 Internal Consultants Who: district curriculum specialists, coaches, administrators, master teachers Plus: work in district every day Delta: may be limited in bringing new skills and strategies to the district

46. © 2006 Success for All Foundation46 Observing Others Who: peers observing peers Plus: maximizes demonstration, modeling, and on-the-job practice with feedback Delta: requires courage

47. © 2006 Success for All Foundation47 Collaborative Peer Groups Who: peer groups who meet regularly to learn and hold each other accountable Plus: allows staff to be fully active participants in their own professional development Delta: finding time is difficult and the temptation to conduct housekeeping chores in meetings is great

48. © 2006 Success for All Foundation48 Professional-Development Format Jigsaw 1. Summarize information from reading. 2. For what component of results-based professional development could this format be used effectively? 3. What other supports would need to be provided along with professional development in this format to ensure teacher success and improve student achievement?

49. © 2006 Success for All Foundation49 Professional Development Based on Student Needs What student needs does the data identify? What are the long-term needs? Are there different needs for different student groups? How should student needs be prioritized?

50. © 2006 Success for All Foundation50 Planning Professional Development Data-Analysis Findings Gap Between Desired Outcome and Current Status Areas of Concern Results-Based Professional Development

51. © 2006 Success for All Foundation51 Results-Based Professional Development ComponentAction StepDateResponsible Knowledge Modeling Low-Risk Practice Feedback On-the-Job Practice with Feedback

52. © 2006 Success for All Foundation52