1. Measuring Post- Release Outcomes Eric Lichtenberger, Ph.D. J. Todd Ogle, Ph.D.

2. Post-Release Outcomes Recidivism Recidivism Employment with earnings being an attribute (magnitude) of employment Employment with earnings being an attribute (magnitude) of employment Educational Attainment Educational Attainment

3. Follow-up Methods Data-Matching Data-Matching Surveying Surveying Mixed-method approach Mixed-method approach

4. Correctional Education Data Educational hierarchy Educational hierarchy Combination of programming Combination of programming Start Date Start Date Exit Date Exit Date Reason for non-completion Reason for non-completion

5. Participation Levels Completers Completers At-Fault Non-completers At-Fault Non-completers No-Fault Non-completers No-Fault Non-completers

6. Planning the Evaluation Two reasons for engaging in evaluation Two reasons for engaging in evaluation Program improvement Program improvement Filling the informational vacuum Filling the informational vacuum The evaluation should be driven by program objectives The evaluation should be driven by program objectives Begin with simple questions about the program Begin with simple questions about the program Identify the information needed to answer the questions and where it will come from Identify the information needed to answer the questions and where it will come from Forming comparison groups Forming comparison groups

7. Improving the Odds of Establishing an Inter-agency MOA or MOU Demonstrate that necessary data security safeguards are in place Demonstrate that necessary data security safeguards are in place Articulate the definitions, goals, and objectives within the evaluation plan Articulate the definitions, goals, and objectives within the evaluation plan Establish reasonable turnaround time for data requests Establish reasonable turnaround time for data requests Start with simple measures and allow them to evolve Start with simple measures and allow them to evolve Establish the significance of the evaluation Establish the significance of the evaluation Be willing to handle raw data Be willing to handle raw data Get key politicians on board Get key politicians on board

8. Theoretical Framework Most correctional education evaluations lack a theoretical framework Most correctional education evaluations lack a theoretical framework Why are we doing what we’re doing and what can we expect to see as a result? Why are we doing what we’re doing and what can we expect to see as a result? However, the justification for most CE programs, as they relate to positive post-release outcomes, can be traced back to Becker’s Economic Theory of Criminality However, the justification for most CE programs, as they relate to positive post-release outcomes, can be traced back to Becker’s Economic Theory of Criminality

9. Economic Theory Applied to CE Offenders are rational actors who choose to commit crimes because their costs and benefits differ from non-criminals Offenders are rational actors who choose to commit crimes because their costs and benefits differ from non-criminals Criminals choose crime because the expected return of illicit activity is greater than the opportunity cost of their time and the impacts of detection Criminals choose crime because the expected return of illicit activity is greater than the opportunity cost of their time and the impacts of detection

10. Conceptual Framework Adapted from Kirschstein and Best (1997) Adapted from Kirschstein and Best (1997) Each outcome measure has three components: geographic scope, timeframe, and precipitating event Each outcome measure has three components: geographic scope, timeframe, and precipitating event Geographic scope and timeframe are often limited by availability and accessibility of data and are part of the operational definition Geographic scope and timeframe are often limited by availability and accessibility of data and are part of the operational definition

11. Defining Recidivism Geographic scope Geographic scope Timeframe Timeframe Qualifying precipitating events (re-arrest, reconviction, re-commitment-could be for a new crime, violation of parole, or either) Qualifying precipitating events (re-arrest, reconviction, re-commitment-could be for a new crime, violation of parole, or either)

12. Recidivism-Related Measures Survival Time (date of release to the precipitating event, if any) Survival Time (date of release to the precipitating event, if any) Most serious new crime related to the precipitating event Most serious new crime related to the precipitating event Resulting sentence or actual time served as a result of precipitating event Resulting sentence or actual time served as a result of precipitating event

13. Potential Evaluation Questions What was the recidivism rate for those in the study group? What was the recidivism rate for those in the study group? What was the average or median survival time for those in the study group? What was the average or median survival time for those in the study group? How many of the recidivists had a precipitating event that could be more serious than their original offense? How many of the recidivists had a precipitating event that could be more serious than their original offense? What is the average resulting new sentence or better yet, time served? What is the average resulting new sentence or better yet, time served?

14. Consider the Following: Group of 100 Correctional Education Participants- Recidivism Rate of 20% after three years Group of 100 Correctional Education Participants- Recidivism Rate of 20% after three years Group of 100 Control Group members (controlling for pre-existing differences)- Recidivism Rate of 20% after three years Group of 100 Control Group members (controlling for pre-existing differences)- Recidivism Rate of 20% after three years What does it say about the Correctional Education Program? What does it say about the Correctional Education Program? What information are we missing? What information are we missing?

15. Defining Employment Geographic scope Geographic scope Timeframe Timeframe Qualifying Precipitating Events (within specific industries or sub-industries, for certain types of employers, within certain locations, earning a certain amount of money, etc.) Qualifying Precipitating Events (within specific industries or sub-industries, for certain types of employers, within certain locations, earning a certain amount of money, etc.)

16. Using UI-Wage Records Inexpensive relative to survey data Inexpensive relative to survey data Feature broad coverage of employment Feature broad coverage of employment Objective data source Objective data source Support longitudinal studies and evaluations Support longitudinal studies and evaluations Include size class and industry of businesses Include size class and industry of businesses Share of employees in an industry that are research subjects Share of employees in an industry that are research subjects Compare company growth rates to state and national trends Compare company growth rates to state and national trends

17. Disadvantages of U-I Wage Records At times you can’t tell if a subject changed employers or when the company’s ownership changed At times you can’t tell if a subject changed employers or when the company’s ownership changed Limited by the state Limited by the state No SSN, or a bad SSN equates to no match No SSN, or a bad SSN equates to no match Does not capture gray-market employment, under-the-table payment arrangements Does not capture gray-market employment, under-the-table payment arrangements 15% lower than actual earnings 15% lower than actual earnings

18. Wage Record Interchange System Department of Labor Program-Employment and Training Administration Department of Labor Program-Employment and Training Administration All States with the exception of California, Connecticut, Hawaii, New Hampshire, and Tennessee All States with the exception of California, Connecticut, Hawaii, New Hampshire, and Tennessee WIA and Social Security Act WIA and Social Security Act

19. Employment Related Measures Dichotomous employment (y/n) Dichotomous employment (y/n) Length of employment (number of consecutive quarters or years employed) Length of employment (number of consecutive quarters or years employed) Breadth of employment (percent of quarters employed) Breadth of employment (percent of quarters employed) Survival time (time to precipitating event) Survival time (time to precipitating event)

20. Potential Evaluation Questions How many CE completers gained post-release employment? How many CE completers gained post-release employment? How many CE completers gained employment before the end of the first quarter following release? How many CE completers gained employment before the end of the first quarter following release? Of those, how many maintained employment for at least two more quarters? Of those, how many maintained employment for at least two more quarters? What is the average number of quarters it takes for CE completers to obtain employment? What is the average number of quarters it takes for CE completers to obtain employment? What is the average number of quarters worked in the study period? What is the average number of quarters worked in the study period?

21. Refining Employment Measures Earned more than a measure of minimum part time employment, minimum full-time employment, over the poverty threshold Earned more than a measure of minimum part time employment, minimum full-time employment, over the poverty threshold Were employed in key industries (growth) Were employed in key industries (growth) Maintained consistent employment with one employer, within one industry, etc. Maintained consistent employment with one employer, within one industry, etc. Exploring aggregate earnings Exploring aggregate earnings

22. Use of Crosswalks Occupational relatedness Occupational relatedness Industrial relatedness Industrial relatedness

23. Consider the Following: Group of 100 Correctional Education Participants- 65% are employed the quarter following release Group of 100 Correctional Education Participants- 65% are employed the quarter following release Group of 100 Control Group members (controlling for pre-existing differences)- 65% are employed the quarter following release Group of 100 Control Group members (controlling for pre-existing differences)- 65% are employed the quarter following release What does it say about the Correctional Education Program? What does it say about the Correctional Education Program? What information are we missing? What information are we missing?

24. Defining Educational Measures Could potentially be the most complex Could potentially be the most complex Geographic scope Geographic scope Timeframe Timeframe Precipitating event (enrollment, degree obtainment, continued enrollment, type of school, type of course, etc.) Precipitating event (enrollment, degree obtainment, continued enrollment, type of school, type of course, etc.)

25. Potential Evaluation Questions What percentage of the study group members enrolled in post-secondary education after release? What percentage of the study group members enrolled in post-secondary education after release? What percentage of the study group earned certificates/degrees after release? What percentage of the study group earned certificates/degrees after release?

26. Student Educational Clearinghouse National bank of post-secondary educational outcomes National bank of post-secondary educational outcomes Charge $0.54 a match Charge $0.54 a match Based on a combination of date of birth and name Based on a combination of date of birth and name 91% of post-secondary students are included 91% of post-secondary students are included

27. Consider the Following: Group of 100 Correctional Education Participants (IYOP Grant)- 15% enroll in post-secondary institutions within a year of release Group of 100 Correctional Education Participants (IYOP Grant)- 15% enroll in post-secondary institutions within a year of release Group of 100 Control Group members (controlling for pre-existing differences)- 15% enroll in post-secondary institutions within a year of release Group of 100 Control Group members (controlling for pre-existing differences)- 15% enroll in post-secondary institutions within a year of release What does it say about the Correctional Education Program? What does it say about the Correctional Education Program? What information are we missing? What information are we missing?

28. Isolating the Impact Statistical techniques such as multiple regression Statistical techniques such as multiple regression Non-completers Non-completers Wait-listed offenders who never participated Wait-listed offenders who never participated Comparison groups limited by key criteria Comparison groups limited by key criteria Exact matching with key variables Exact matching with key variables Propensity score matching Propensity score matching

29. Propensity Score Matching Select the variables that could have an impact on the likelihood someone will complete a CE program Select the variables that could have an impact on the likelihood someone will complete a CE program Model is created based on the characteristics of the CE completers and applied to a group of non-CE participants Model is created based on the characteristics of the CE completers and applied to a group of non-CE participants Use a group of non-participants whose only difference is that they haven’t taken the given CE program Use a group of non-participants whose only difference is that they haven’t taken the given CE program

30. Considerations Define the outcomes in terms of the questions you want answered Define the outcomes in terms of the questions you want answered Start with more simple measures of post-release outcomes Start with more simple measures of post-release outcomes Disaggregating outcome information (by program, location, etc.) can help pinpoint potential problems Disaggregating outcome information (by program, location, etc.) can help pinpoint potential problems

31. Summary A successful evaluation of post-release outcomes begins with planning A successful evaluation of post-release outcomes begins with planning Well-planned evaluation can provide information for communication with internal and external constituencies Well-planned evaluation can provide information for communication with internal and external constituencies Isolating programmatic impact of post-release outcomes justifies program costs Isolating programmatic impact of post-release outcomes justifies program costs Evaluate your programs using your outcomes or someone else will Evaluate your programs using your outcomes or someone else will

32. Contact the Authors Eric Lichtenberger Eric Lichtenberger elichten@vt.edu elichten@vt.edu 540-231-2549 540-231-2549 Todd Ogle Todd Ogle todd.ogle@vt.edu todd.ogle@vt.edu 540-231-4228 540-231-4228