1. California Standards Test and CAHSEE Correlation Use of Student Data for Targeted Preemptive Intervention November 1, 2006 Dr. Janis Fries-Martinez, Principal Gerardo Loera, Assistant Principal Polytechnic High School Los Angeles Unified School District 2

2. In This Presentation You Will Learn: How to accurately identify 10 th grade students who need intervention for the CAHSEE before they take it in the spring How to identify a target group of students to strategically raise the percentage of students scoring Proficient or Above on the CAHSEE in order to meet AYP requirements How to predict, with significant amount of certainty, which 10 th grade students will pass and fail the CAHSEE on their initial try

3. No Child Left Behind and State Accountability Criteria - CAHSEE AYP and API are dependent in large part to CAHSEE scores  AYP: Annual Measurable Objectives are measured mainly by success of 10 th grade students taking the CAHSEE the 1 st time Problem: High failure rate on CAHSEE Problem: Not enough students scoring Proficient or Above on CAHSEE CAHSEE GraduationAPIAYP

4. Guiding Questions How do we effectively target students for pre- emptive intervention for success on the CAHSEE before they initially take it? How do we proactively identify students that need additional support? Do we know which students are likely to pass the test or get a proficient score ?

5. A passing score on the CAHSEE is:  350 or greater on the English Language Arts portion of the test AND  350 or greater on the Mathematics portion of the test A proficient score is:  380 or greater on the English portion of the test  380 or greater on the Mathematic portion of the test What are Passing and Proficient Scores?

6. Some Possible Attributes Contributing To Success On The CAHSEE? Grades/Marks Demographics  Socioeconomic status  Learning environment CELDT Scores Language Classification Special Education Status CST Scores Periodic Assessments Etc.

7. Studied CST/CAHSEE Correlation at Different High Schools Test Groups: Poly HS: Class of 2006 Poly HS: Class of 2007 Santee HS: Class of 2007 LAUSD Class of 2008 (37,000+ scores) LAUSD Class of 2007 (36,000+ scores)

8. Polytechnic High School Class of 2007 9 th Grade ELA CST vs. 10 th Grade ELA CAHSEE – Matched Scores Note: 800+ scores included Source: Secondary Student Information System

9. South LA High School #1 - Class of 2007 9th Grade ELA CST vs. 10th Grade ELA CAHSEE – Matched Scores

10. LAUSD Class of 2008 - 9th Grade ELA CST vs. 10th Grade ELA CAHSEE – Matched Scores Note: 37,000+ scores included Source: School Information Branch

11. LAUSD Class of 2008 - 9th Grade Algebra I CST vs. 10th Grade Math CAHSEE – Matched Scores Note: 23,282 scores included Source: School Information Branch

12. LAUSD Class of 2008 - 9th Grade Geometry CST vs. 10th Grade Math CAHSEE – Matched Scores Note: 8,410 scores included Source: School Information Branch

13. LAUSD Class of 2007 - 9th Grade MATH CST vs. 10th Grade Math CAHSEE – Matched Scores Note: 36,190 scores included Source: School Information Branch

14. Math CAHSEE Predictors Mathematics – Algebra I Model 9th Grade CST score of 271 or higher on Algebra I CST predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score of 334 or higher predicts a Proficient score of 380 or higher on the CAHSEE Mathematics – Geometry Model 9 th Grade CST score of 214 or higher predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score of 280 or higher predicts Proficient score of 380 or higher on the CAHSEE (AYP)

15. Summary of Predictors English Language Arts Model Based on LAUSD Class of 2008 (37,000+ scores) 9 th Grade English CST score of 292 or higher predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score of 346 or higher predicts Proficient score of 380 or higher on the CAHSEE (To meet AYP)

16. Data from the LAUSD School Information Branch – Class of 2008 Model: 292 ELA CST = 350 ELA CAHSEE Predicted Results PassFail Actual Results Pass240712914 Fail35498212 This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a passing score or failing score on the English portion of the CAHSEE based on the 9 th grade CST is correct 83.4% of the time. ELA – “Reliability” Matrix for predicting Passing CAHSEE ELA score – 83.4% accurate 26985 Total 11761 Accuracy

17. ELA – “Reliability” Matrix for predicting Passing CAHSEE ELA Proficient or Above score – 84.6% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 346 ELA CST = 380 ELA CAHSEE Predicted Results Prof/AboveBelow Prof Actual Results Prof/Above112882135 Below Prof384621477 This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore this model is correct 84.6% of the time.

18. Algebra I - “Reliability” Matrix for predicting Passing CAHSEE Math Passing score – 72.9% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 271 Alg. I CST = 350 MATH CAHSEE Predicted Results PassFail Actual Results Pass112312930 Fail33835737 This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a passing or failing score on the Mathematics portion of the CAHSEE based on the 9 th grade CST is correct 72.9% of the time.

19. Algebra I – “Reliability” Matrix for predicting Passing CAHSEE Math Proficient or Above score – 85.1% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 334 Alg. I CST = 380 MATH CAHSEE Predicted Results Prof/AboveBelow Prof Actual Results Prof/Above26542742 Below Prof72617159 This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a score on the Mathematics portion of the CAHSEE based on the 9 th grade CST is correct 85.1% of the time.

20. Geometry - “Reliability” Matrix for predicting Passing CAHSEE Math Passing score – 88.7% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 214 Geom. CST = 350 MATH CAHSEE Predicted Results PassFail Actual Results Pass737468 Fail88682 This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a passing score or failing score on the Mathematics portion of the CAHSEE based on the 9 th grade Geometry CST is correct 88.7% of the time.

21. Geometry – “Reliability” Matrix for predicting Passing CAHSEE Math Proficient or Above score 82.1% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 334 Geom. I CST = 380 MATH CAHSEE Predicted Results Prof/AboveBelow Prof Actual Results Prof/Above4539721 Fail7812369 This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a Proficient or Above score on the Mathematics portion of the CAHSEE based on the 9 th grade Geometry CST is correct 82.1% of the time.

22. Math (ALL) - “Reliability” Matrix for predicting Passing score on CAHSEE Math – 72.7% accurate Data from the LAUSD School Information Branch – Class of 2007 Model: 268 Math CST = 350 MATH CAHSEE Predicted Results PassFail Actual Results Pass190703634 Fail62377249 This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a passing score or failing score on the Mathematics portion of the CAHSEE based on the 9 th grade Math CST is correct 72.7% of the time.

23. Math (ALL) – “Reliability” Matrix for predicting Passing CAHSEE Math Proficient or Above score 81.5% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 326 Math CST = 380 MATH CAHSEE Predicted Results Prof/AboveBelow Prof Actual Results Prof/Above56525127 Below Prof156423974 This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a Proficient or Above score on the Mathematics portion of the CAHSEE based on the 9 th grade Math CST (regardless of which CST test was taken such as Algebra I, Geometry, General Math etc.) is correct 81.5% of the time.

24. Summary of Predictors Mathematics Model Based on LAUSD Class of 2007 (36,190 scores) 9 th Grade CST score in Math of 268 or higher predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score in Math of 326 or higher predicts a 380 or higher on the CAHSEE (AYP) English Language Arts Model Based on LAUSD Class of 2008 (37,000+ scores) 9 th Grade English CST score of 292 or higher predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score of 346 or higher predicts Proficient score of 380 or higher on the CAHSEE (AYP)

25. For Program Improvement Schools:  A Review of the 2006 AYP Progress Report needs to be made in order determine which subgroups, if any, fell short of meeting the Annual Measurable Objectives (AMO) Recall that the AMOs are directly associated with the percentages of 10 th grade students that score Proficient or Above on the ELA and Math portions of the CAHSEE Deciding Which Students To Target – Possible Action Plans For Applying the ELA Model

26. For Program Improvement Schools (cont.):  Once a subgroup has been identified, an SSIS extract can be performed to create a list of students needing pre-intervention The extract should produce a list of 10 th grade students that scored less than a 346 on their English 9 th Grade CST. This list should be sorted in decreasing order by their English 9 th Grade CST scaled score. Deciding Which Students To Target – Possible Action Plans For Applying the ELA Model

27. For Program Improvement Schools (cont.):  Once a subgroup has been identified, an SSIS extract can be performed to create of list of students needing pre-intervention The extract should produce a list of 10 th grade students that scored less than a 326 on their Math 10 th Grade CST. This list should be sorted in decreasing order by their Math 9 th Grade CST scaled score. Deciding Which Students To Target – Possible Action Plans For Applying the Math Model

28. For Program Improvement Schools (cont.):  The resulting lists are prioritized lists of students that will benefit from pre-intervention and would most likely not score a Proficient or Above on the CAHSEE without additional support. Deciding Which Students To Target – Possible Action Plans For Applying the ELA Model

29. How to identify 10 th grade students that will probably fail one or both parts of the CAHSEE?  Perform an SSIS extract for 10 th Grade students that scored less than a 268 on their 9 th Grade Math CST  Perform an SSIS extract for 10 th grade students that scored less than a 292 on their 9 th Grade English CST Deciding Which Students To Target – Possible Action Plans For Applying the Predictive Model

30. Cynthia Lim in the LAUSD School Information Branch has offered to provide schools with technical support on identifying students at- risk of not passing the CAHSEE or not getting a Proficient or Above score. Cynthia.lim@lausd.net Deciding Which Students To Target – Technical Assistance

31. CAHSEE Preemptive Intervention Case Study – Polytechnic High School 2006 Master schedule permitted for approximately 170 seats for CAHSEE preemptive intervention in Spring of 2006 Selected a window of scores predicted to be 300-370 Based on student performance this resulted in a target range as follows: For ELA: CSTs of 212 - 328 For Math: CSTs of 164 - 308

32. On average, non prepped 10 th Graders matched the predicted values. Math predicted mean score: 347.5 Math actual mean score: 348.5 ELA predicted mean score: 353.0 ELA actual mean score: 352.2 CAHSEE Preemptive Intervention Case Study – Polytechnic High School 2006 - Results

33. ELAMath # In preemptive preparation 130166 # Predicted to pass108118 # Actual did pass120143 Increase+12+25 % Increase11.1%+21.2 # Predicted proficient4816 # Actual Proficient6167 Increase+13+51 % Increase27.1%319% CAHSEE Preemptive Intervention Case Study – Polytechnic High School 2006 - Results

34. CAHSEE Preemptive intervention works! Preparation was a cost effective means of positively impacting the study group. CAHSEE Preemptive Intervention Case Study – Polytechnic High School 2006 - Results