1 Evaluating the Orange County School Mobile Assessment and Response Team (SMART) Association for Criminal Justice Research, California 63rd Semi-Annual Meeting Sacramento March 30th-31st, 2006 Sandy Hilger, OC Probation

2 Juvenile Justice Crime Prevention Act (JJCPA) JJCPA was passed in 2000, creating a stable funding source for local juvenile justice programs The goal is to support development of local juvenile justice programs based on evidence-based practices. In the first four years a total of $470.2 million was made available to 56 counties for 193 collaborative efforts along a continuum of responses to juvenile crime – prevention, intervention, supervision, treatment and incarceration. During the 4th year, alone, over 110,500 youth were served by the programs.

3 Evaluating the JJCPA Programs The JJCPA requires the Corrections Standards Authority (former BOC) to submit an annual report to the Legislature on the results of the six mandated juvenile justice outcomes. The most recent statewide report (2004), states that “the outcomes indicate that the programs are making a significant difference in curbing juvenile crime and delinquency.” It notes: An average of 21.8% of program juveniles were arrested versus 32.5% of reference group juveniles; An average of 18.2% of program juveniles were incarcerated versus 23.4% for reference group juveniles; and An average of 56.3% of program juveniles completed court- ordered community service versus 39.4% for reference group juveniles.

4 Outcome Measures for SMART  To meet the reporting requirements, CSA collects outcome measures from each county for every program. There are several required outcome measures for all programs as well as local measures established by the counties. Each outcome is measured against a comparison group.  Outcomes areas for SMART include:  Arrests of participants during a one-year follow-up period (mandated)  Incarcerations of participants during a one-year follow-up period (mandated)  Number of juvenile arrests in the service areas  Perception of school safety as self-reported on the Healthy Kids Survey  School-Related Crime Incidents (Reports to California Dept. of Education)  Number of Calls for Service and Number of Completed Threat Assessments

5 Outcomes Reported for SMART Overview of major points: 1. The state-mandated outcomes for SMART were less than meaningful for purposes of local outcomes. 2. Several of the outcomes showed significant differences but no evidence that the differences were due to implementation of SMART. 3. The most interesting outcomes pertained to the decisions and activities of the SMART team, including: a. The proportion of calls for service that became full threat assessments. b. Further actions taken as a result of each call for service such as referrals to counseling, citations for tobacco or alcohol use, confiscation of weapons, arrests, and mental safety holds.

6 Were fewer juveniles arrested after contact with SMART? Outcome Measure #1: Percent of Juveniles Arrested. Were fewer juveniles arrested after contact with SMART? The proportion of arrests among juveniles contacted by SMART increased significantly within the year following the juvenile’s contact with the SMART team.  2 (1, N = 2052) = 5.70, p <.05 However, using Phi as as Measure of Association we see a negligible association between SMART contact and arrest status. (  =.052) It is possible that the rise in number of arrests may have been associated with other factors such as the increased age of the participants.

7  There was no significant change in the proportion of incarcerations among juveniles during the year following contact with SMART.  2 (1, N = 2052) = 1.57, p >.05  In addition, there was negligible association between SMART contact and incarceration status. (  =.03) Outcome #2: Incarcerations per SMART Juvenile Were fewer juveniles incarcerated after contact with SMART?

8  There was a significant decline in the number of arrests in the service area (Test for Proportional Differences, Z = , p. <.001) Outcome #3: Arrests in Service Area Did the rate of juvenile arrests in the service area decrease?

9 Outcome #4 : Perception of School Safety Did juveniles feel safer as a result of SMART implementation?  School Safety was measured according to the percentage of students who reported feeling safe on the Healthy Kids Survey administered by the schools. No significant difference was found on this measure Pre versus Post SMART implementation  2 (1, N = 9270) = 2.83, p >.05 However, the percent of students who reported feeling safe at school was high even prior to SMART (93% versus 94% after SMART)

10 Outcome #5: Number of Crime Incidents at School Were there fewer reports of school crime after SMART implementation?  The number of crime incidents, as reported in School Incident Reports, indicated a significant decrease after the implementation of SMART, (Test for Proportional Difference, Z = 9.45, p <.01)

11 #6: Number of Call for Service and Threat Assessments Did the proportion of calls for service resulting in full threat assessments change over the years of SMART?  The proportion of calls for service resulting in full threat assessments during decreased significantly between 2002/2003 and 2004/2005  2 (1, N = ) = p <.01  A test for association showed a moderate association between proportion of calls resulting in threat assessments and year of SMART ( 

12 Other Factors Worthy of Consideration: Number of Weapons Confiscated as a Result of School Threat Assessments From 2001 to 2005, the SMART team confiscated weapons per year. During the past two school years, for example, they confiscated 68 knives, 23 firearms (at least 2 were loaded handguns), 3 explosives and 20 other weapons.

13 Other Factors Worthy of Consideration Between 2001 and 2005, the SMART team responded to over a thousand calls for service, and had contact with over 1300 juveniles. What were the resulting actions for those kids? u 43% were referred to community counseling services u 27% were arrested u 19% required no further action (outside of school services) u 5% were cited for tobacco or alcohol abuse u 1.5% required a W&S 5150 hold for mental safety evaluation

14 The Future of SMART and the Importance of Experience JJCPA funding allowed for SMART to be implemented Each year, the support has been reduced, resulting in loss of personnel The team has made changes over the years as they gained experience and training in the area of assessing and managing juvenile behavior (warning signs and/or impending threats of safety) Although previous losses in staff and personnel have been manageable, SMART is currently facing reduction in staff that may threaten the existence of the program Although measures such as number of calls of service, number of weapons confiscated, and other process outcomes have allowed us to argue for the value of the program, it is not possible to know the number of dangerous incidences that have been avoided by the availability of this team…whether the danger would have come from the students themselves, or over-reaction by inexperienced officers.

15 Sources of Information JJCPA California Corrections Standards Authority (Board of Corrections) 2004 Report: Bureau of Justice Statistics: Indicators of School Crime and Safety, U.S. Dept. of Education: A guide to Safe Schools (see for an extensive list of references) Safe Schools: a New Approach to Create a Non-Violent Campus (see list of references)

16 Contacts Orange County Probation Department: Sandy Hilger, Research Analyst, Research Division (714) Orange County Sheriff Department: Sgt. Brad Virgoe, SMART, Orange County Sheriff (714)