Considerations for Establishing Baseline and Setting Targets for Indicators C3 and B7 Kathy Hebbeler, Lynne Kahn, Christina Kasprzak ECO/NECTAC June 16, 2009

Two related calls Last Tuesday, June 9, 2009 on “Review of Summary Statements” –Powerpoints on the ECO website Today, “Considerations for Setting Targets” 2

What we’ll cover today Two strategies for examining data –Data quality –Potential for program improvement Parameters, guidance for target settings from OSEP 3

Can you trust the data? Begin by identifying outliers Examples: look at the percentages reported for category a and category e across local programs 4

Percentages reported in category “a” across 30 local programs 5

Remove the outliers State percentage for “a” with all data= 3.9% Revised percentage for “a” with outliers removed= 2.4% 6

Percentages reported in category “e” across 30 local programs 7

Remove the outliers State percentage for “e” with all data= 32.1% Revised percentage for “e” with outliers removed= 27.7% 8

Example of data with outliers removed Progress Category Original %Clean % a 42 b 1517 c 2730 d 31 e 2420 Sum St Sum St Clean data (without the outliers) may be a more accurate picture of where you are starting 9

Suggested strategy Analyze your data with your local LEA/program outliers included and excluded so you can gauge the impact they are having on your state level data.

Note Note Note Consider clean data when deciding about reasonable targets, BUT Turn in the original data to OSEP in the SPP report! You can discuss the clean data in the rationale for your targets. 11

Which local programs can be targeted for program improvement? Compare the summary statement data by local program to identify which programs have the most potential for improvement. 12

Summary Statement Percentages by Local Program 13

Considerations What do you know about the programs/LEAs with the least and the most progress in the summary statements? i.e. the programs w/ –the lowest and highest percentages of children at age expectation at exit –the lowest and highest percentage of children making greater than expected gains 14

Examples of Key Questions Are the children similar at entry? Are the higher performing programs/LEAs participating in special projects? e.g. a state initiative, TACSEI or CELL? Are there systems issues in lower performing programs/LEAs that would explain differences in outcomes? e.g. personnel shortages 15

Bottom-line Question Could either system or practice focused improvement activities targeted toward the lowest performing programs/LEAs improve the child outcomes? 16

The Math of Target Setting How much would the data change if the lowest local programs moved toward the mean? Improvements in the lowest programs will result in improvement in your statewide data Experiment with your data to determine what targets are reasonable in your state 17

Timelines In Feb, 2010, in SPP format: –Baseline –Targets for 2 reporting years –Improvement activities for 2 reporting years In Feb, 2011 and 2012, in APR format –Actual data, progress and slippage, etc. –Local reporting of [summary statement %s] 18

Questions and comments? 19

Outcomes Conference June 22 and 23, Bethesda, MD Resources at The-ECO-Center.org 20