Challenges in Determining Whether Treatment Programs are Effective How do we know what works? Lewis A. Davis Judge of the Superior Court County of Contra Costa ldavi@contracosta.courts.ca.gov

“Evidence-based” practices A 2016 review of the entirety of the Codes of California reveals that: The term “evidence-based” practices is used 31 times 9 of those mentions pertain to the Penal Code 7 of those mentions reference the Welfare and Institutions Code (including both W/I §602 (juvenile delinquents) and W/I §300 (neglected and abused kids)

“Recidivism” The term “recidivism” is used 67 times Not surprisingly, 46 of those mentions pertain to the Penal Code 13 of those mentions pertain to the W/I Code (most of those reference W/I §602 juvenile delinquents)

Definitions of “evidence-based practices” The most “comprehensive” definition of “evidence-based practices”– “Evidence-based practices” means supervision policies, procedures, and programs, and practices demonstrated by scientific research to reduce recidivism among individuals under probation, parole, or postrelease community supervision.” PC §§1170.05(f)(2), 1170.06(f)(2), 17.5(a)(9), 3450(b)(9). “’Evidence-based practice” means a decisionmaking process that integrates the best available scientifically rigorous research, clinical experience, and individual’s characteristics.” W/I Code §4686.2(d)(3).

Definitions of “Recidivism” Most of the statutory sections that mention “recidivism” do so without defining it. The most “comprehensive” definitions of “recidivism” are listed below. 1. “A community recidivism and crime reduction service provider shall provide services that are designed to enable persons to whom the services are provided to refrain from engaging in crime, reconnect with their family members, and contribute to their communities. “ PC §1233.10 (C)(2).

Definitions of “Recidivism” 2.“Recidivism” means a conviction of a new felony or misdemeanor committed within three years of release from custody or committed within three years of placement on supervision for a previous criminal conviction.” PC § 6046.1(d).

Problems with defining “recidivism” Is recidivism to be measured by either arrest or conviction (sustained petition)? Should the conduct be limited to felony offenses or should misdemeanors be counted as well? (3) If misdemeanors are to be included should it be all misdemeanors or only violent ones?

Problems with defining “recidivism” (4) What period of time should an offender’s subsequent behavior be monitored – many studies tout positive results within six months or one year as proof that a program is effective; (5) Has the program employed an experimental design by which program participants are randomly assigned to particular programmatic approaches? and (6) Does the program design contemplate the use of a control group?

“Evidence-based practices” in juvenile court "Evidence-based" is defined as a program or practice that has been tested in: (1) heterogeneous or intended populations with multiple randomized, or statistically controlled evaluations, or both; or (2) one large multiple site randomized, or statistically controlled evaluation, or both, AND (3) where the weight of the evidence from a systemic review demonstrates a sustained reduction in recidivism.   "Evidence-based" also means a program or practice that can be implemented with a manualized set of procedures to allow successful replication in California and, when possible, is determined to be cost-beneficial.

“Recidivism” in juvenile court  (1) A sustained finding of a new felony or misdemeanor offense, or (2) A sustained violation of probation for an offense that would have been recognized as a felony or a misdemeanor if it had been filed as a new petition, AND (3) Was committed within three years of the following: (a) release from custody, or (b) placement on supervision for a previous sustained finding of a felony or misdemeanor offense, or (c) placement on supervision for a prior sustained probation violation that would been recognized as a felony or a misdemeanor if it had been filed as a new petition.

“Evidence-based practices” in adult court "Evidence-based" is defined as a program or practice that has been tested in: (1) heterogeneous or intended populations with multiple randomized, or statistically controlled evaluations, or both; or (2) one large multiple site randomized, or statistically controlled evaluation, or both, AND (3) where the weight of the evidence from a systemic review demonstrates a sustained reduction in recidivism. "Evidence-based" also means a program or practice that can be implemented with a manualized set of procedures to allow successful replication in California and, when possible, is determined to be cost-beneficial.

“Recidivism” in adult court (1) A conviction of a new felony or misdemeanor offense, or (2) A sustained violation of probation for an offense that would have been recognized as a felony or a misdemeanor if it had been filed as a new complaint, AND (3) Was committed within five years of the following:  (a) release from custody, or (b) placement on supervision for a previous criminal conviction, or (c) placement on supervision for a prior sustained probation violation that would have been recognized as a felony or a misdemeanor if it had been filed as a new complaint.