Presentation on theme: "DOING IT RIGHT Risk Assessment in Juvenile Justice April 25, 2013"— Presentation transcript:
1 DOING IT RIGHT Risk Assessment in Juvenile Justice April 25, 2013 Gina M Vincent, PhD Associate Professor, UMass Medical School National Youth Screening & Assessment Project (NYSAP)
2 Leads a national movement State-based juvenile justice coalitions and organizationsLaws, policies and practices that are fair, equitable and developmentally appropriate for all children, youth and familiesPhoto: Moriza
3 Risk Assessment in Juvenile Justice: A Guidebook for Implementation Combines empirical evidence with the consensus of experts from three panels & multiple reviewers.Experts/Scholars on Risk AssessmentAdvisory Group (practitioners)Legal StakeholdersFocuses on risk assessment at probation or probation intakeLayoutExecutive SummaryExplanation of risk assessment concepts8 steps of implementationCD with policy and other document templates
4 OutlineWhy is knowledge of a youth’s risk-level important for dispositional decisions and case management?What is a risk assessment tool?Why is it better than current practice?What makes a tool evidence-based?What can risk assessment do for you if it is properly implemented?Implementation Issues
5 Why Is knowledge of a youth’s risk-level Important? The next set of slides show some results from the Risk Assessment in Juvenile Probation: Implementation Study. This was a pre-post study of the impact of implementing an evidence-based risk assessment tool (the YLS/CMI or the SAVRY) in six probation offices. A sample of adjudicated youth was selected before a tool was implemented (pre) and compared to a sample of youth who were adjudicated after a tool was implemented (post) in each office. The samples of youth varied from 100 to 250 depending on the probation office. Youth were tracked for up to 13 months to record their disposition, any placements, services received, and their levels of supervision.
6 Guiding Principles: Research Evidence There is emerging consensus on characteristics of effective programming for young offenders:Punitive sanctions without effective services do not have a significant effect on re-offending (Gatti et al., 2009).Most low-risk youth are unlikely to re-offend even if there is no intervention (Lipsey, 2009). But mixing them with high risk youth can make them worse.When services are matched to youth’s “crime-producing” (criminogenic) needs, the lower the chance of repeat offending.The goal is to have the right services for the right youth.Punitive sanctions without services has no impact on re-offending and actually costs more $$ than diversion. The 20-year study tracked youth from age 10 to age 17, then tracked their arrest records in adulthood. Those who entered the juvenile-justice system even briefly - for example, being sentenced to community service or other penance, with limited exposure to other troubled kids - were twice as likely to be arrested as adults, compared with kids with the same behavior problems who remained outside the system. Being put on probation, which involves more contact with misbehaving peers, in counseling groups or even in waiting rooms at probation offices, raised teens' odds of adult arrest by a factor of 14. Study: Gatti, U., Tremblay, R. E., & Vitaro, F. (2009). Iatrogenic effect of juvenile justice. Journal of Child Psychology & Psychiatry, 50, doi: /jThe large meta-analysis by Lipsey showed that services that focused on high risk youth were more effective in reducing recidivism and had little to no effect for low risk youth. Study: Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims & Offenders, 4, doi: /When services are matched to youth’s crime producing needs – greater chance of reducing reoffending. Better than merely piling on services.Also note that merely piling services onto youth unlikely to have a positive impact – could be detrimental – definitely not cost effective. Instead we want the right services for the right youth.Research suggests justice agencies will have more success if they apply the Risk-Needs-Responsivity Principle to case management. This means formal processing and case management should be commensurate with ones level of risk for reoffending and should address the youth’s specific criminogenic risk factors. WE want the right interventions and the right services for the right youth.
7 Results of Cost/Benefit Research: Benefits Per Dollar Invested For every $1.00 spent on the following services, you save:Functional Family Therapy: $28.34Multisystemic Family Therapy: $28.81Multidimensional Treatment Foster Care: $43.70Adolescent Diversion Project: $24.92Juvenile Boot Camps: $0.81Scared Straight: -$ (NET LOSS)Suggested citation: Steve Aos, Marna Miller, andElizabeth Drake. (2006). Evidence-Based Public PolicyOptions to Reduce Future Prison Construction, CriminalJustice Costs, and Crime Rates. Olympia: WashingtonState Institute for Public Policy.Link to full text of that report available at:
8 Risk-Needs-Responsivity (RNR) Approach to Case Management Risk – Match the intensity of the intervention with one’s level of risk for re-offendingTells us ‘Who’ to targetUseful for level of supervision/intensity of services/ placement & dispositionNeed – Target criminogenic needs (or dynamic risk factors)Tells us ‘What’ to targetProvide only services for areas where youth have the highest needsResponsivity – Match the mode & strategies of services with the individualAn evidence-based method for using risk assessment in case planning is known as the Risk-Needs-Responsivity (RNR) approach. This slide describes the approach.8
9 Matching Services to Criminogenic Needs Can Have a Large Impact (Vieira et al., 2009) % Re-offendedThis slide illustrates the potential impact of matching services to youths’ needs on reoffending. These are results of an archival study of 122 young offenders who received the YLS/CMI by a clinician/social worker and then were tracked to see if the probation officers gave the youths services that were in response to the clinician-identified criminogenic need factors.Good match = 75% or more of the youths needs were addressed by a serviceMed match = 26% - 74% of needs addressed by a servicePoor match 25% or less needs addressed by a serviceThis slide shows how the service-to-needs match can work. There was a substantially lower recidivism rate among youths who had a good match between the services they received and their needs than for the other youths.The study also looked at the pure number of services youths received and there was an association with recidivism but not nearly as strong as what this chart shows.Study: Vieira, T. A., Skilling, T. A., & Peterson-Badali, M. (2009). Matching court-ordered services with treatment needs. Criminal Justice and Behavior, 36(4), doi: /Match based on # of Services Given in Response to aYouth’s Criminogenic Needs
10 Risk Changes Across Adolescence, For Most Per capita arrest rates and self-report violent behavior go up through teens and back down after late teens.3 out of 4 youths arrested for violent offenses are not repeat violent offenders in adulthood.Moffitt – check how violence measuredLife-course Persistent Offenders (Moffitt, 1993): 6 – 8%, Offending and aggression start in preschool and persist throughout adulthood,The view is that this risk may be inherited or acquired from neuropsychological variationAdolescent-Limited Offenders: 75% - Violence begins and ends during adolescence, Police contact is typically not before age 13. Over 75% of these youths will desist during adolescence (Moffitt, 1993; Farrington, 1986)Point Being: The majority of serious adolescent offenders will stop by early adulthood. Offending is not chronic for most. Good Risk/needs assessment is important to weed out the youths who may have committed a serious act but aren’t necessarily high risk. Reassessment is important.
11 Why are Objective, Validated Instruments Better Than Current Practice? What do we do now?What is the accuracy?First step is valid identification as to who is high versus low risk. Many states/jj programs have been adopting risk assessment tools in part because of the suggestion in the Juvenile Justice Delinquency Prevention Act, and recent research. Generally people recognize it is a good thing to do. It may lead to lower reoffending ratings and protection to public safety while maximizing effective use of interventions and services.POs for example can use standardized risk assessment tools to help them make recommendations to the courts about dispositions– for example, does the youth need secure custody or can they be safely managed in the community? To the extent these decisions are based on threat to public safety that is. There is some evidence that putting low risk people into intensive interventions with high risk youth can may make them worse.Identification of high versus low risk youth should proceed intervention planning. Because Identification of high risk people can lead to case planning that is commensurate with one’s risk. Ideally, you want to use risk assessments that will also identify the factors in a youths life that are likely driving their reoffending (crime-producing factors or criminogenic needs) so services can be put in placeLower risk youth do not need as much monitoring while on supervision.However, I would argue that we want to conduct these assessments in a manner that guides intervention efforts to reduce a youth’s risk. This is about service delivery and risk reduction. This comes down to appropriate implementation
12 Elements of an Evidence-based risk assessment tool The next set of slides show some results from the Risk Assessment in Juvenile Probation: Implementation Study. This was a pre-post study of the impact of implementing an evidence-based risk assessment tool (the YLS/CMI or the SAVRY) in six probation offices. A sample of adjudicated youth was selected before a tool was implemented (pre) and compared to a sample of youth who were adjudicated after a tool was implemented (post) in each office. The samples of youth varied from 100 to 250 depending on the probation office. Youth were tracked for up to 13 months to record their disposition, any placements, services received, and their levels of supervision.
13 What is a Risk Assessment Tool? Risk = risk for serious delinquent or violent offendingA risk for reoffending or violence assessment tool is an instrument developed to help answer the question: “Is this youth at relatively low or relatively high risk for reoffending or engaging in violent behavior?”Brief risk assessmentComprehensive risk assessmentSome, also address “What is possibly causing the youth to be at low or relatively high risk for reoffending?”First I should explain what is a risk assessment tool?
14 What Does the Risk Level Mean? Risk is relativeLow riskFew risk factors, or few salient risk factorsLow intensity management/supervision sufficientIf left alone or with minimal management, would likely not reoffendHigh riskMany risk factors, or some critical risk factorsHigh intensity management/supervision necessaryIf left alone or with minimal management, would likely reoffendModerate risk: neither high nor low riskWhy low, moderate, and high?Clinician preferenceLess susceptible to bias than frequencies or probabilitiesNo false precision (but has fuzzy clarity)
15 Example Actuarial Tool 42 Risk Items8 Domains- Family- Attitude/orientation+ StrengthsItems rated present/ absent using interview + all available infoBoth tools contain
16 Example Structured Professional Judgment Tool 24 Risk Items- 10 Static- 14 Dynamic+ 6 Protective ItemsItems rated a on 3-pt scale using interview + all available infoThe SAVRY is just one example of a risk assessment – one of the best our current science has to offer. Includes CU traits and other factors known to increase the likelihood that youth will reoffend. Family, subs abuse, peersAccuracy at an acceptable level according to convention in psychology and psychometrics – for describing the average youth. Will always be cases that are exceptions – Columbine - not the garden variety offender – targeted violence or terroristic violence (principled) or sexual – not going to be identifiedRisk assessments can only maintain accuracy if conducted with youth who already came into contact with the law or forensic system due to some type of offenseWill id your typical serious young offender based on what we know of general violence or offending. Risk for Targeted violence – violence directed toward a single individual or group due to some principle or some form of MH – unlikely to be identified. Terrorism, columbine, sniper
17 Definition of Terms: Risk Factor Risk factor: anything that increases the probability that a person will re-offend:1. Static Risk Factors – do not changeExamples:Early Onset of violent behaviorEarly Onset of arrestsNumber of past delinquent actsOnset of Substance UseRisk factors are not necessarily causesEx – history of child abuse often seen as a risk factor – predictive of risk for future offending but some abused kids offend most don’t‘ – why? Reslliency – name some protective factors
19 Dynamic Risk Factors Antisocial Peers Substance Use/AbuseIf it has a direct effect on their delinquent activity and is outside of the norm for adolescencePoor School/Work AchievementAntisocial PeersNeeds that are NOT Criminogenic do NOT belong in risk assessment tools (at least not for calculating a risk score) :Some mental health variablesSelf-mutilationLearning DisabilityThere is a lot of information below. Just cover pieces that seem to be most important for the specific audience.Family: There is a large body of research that has linked ineffective parenting practices with violent behavior. For example, results from McCord’s (1979) 20 year follow up study, demonstrated that poor parental supervision and aggressive discipline predicted crimes against a person well into their forties. One explanation for this relationship is that children and youth who are treated aggressively may be more likely to view this type of behavior as acceptable, whereas those who receive no supervision or inconsistent guidelines may be unaware of the parameters of acceptable behavior (Department of Health and Human Services, 2000). There is also evidence that a lack of parent-child interaction may increase the youth’s likelihood for future violence, specifically when parent-child communication and parental involvement is absent during adolescence (Williams, 1994).Peers: Peer rejection has also been implicated in the later expression of aggression among children (see US Department of Health, 2001, for a review). For example, children and adolescents who report having weak social ties and experiencing rejection by conventional peers have been found to be at an elevated risk for violence (Farrington, 1989; Lipsey & Derzon, 1998; Saner & Ellickson, 1996). The fact that a large proportion of crimes are committed with peers in adolescence further reinforces the need to examine the influence of peers (Zimring, 1998). Finally, gang association is a dimension of peer socialization that has demonstrated an independent effect on violence in adolescence (Battin, Hill, Abbot, Catalano, & Hawkins, 1997; Thornberry, 1998). This item will eventually be rated in accordance with the level of exposure and/or bonding to antisocial or delinquent peers, as well as aspects of unpopularity and/or victimization.Community: During adolescence, measures of community disorganization, such as the presence of neighborhood violence (Durant et. al, 1994; Singer et al., 1995; Widom, 1989); accessibility to firearms and drugs (Maguin et al., 1995); the number of neighborhood adults who are involved in antisocial behaviors and violence (Elliot, 1989; Maguin et al., 1995); high levels of social disorganization (Osgood & Chambers, 2000; Sampson & Lauritsen, 2994); exposure to prejudice (McCord & Ensminger, 1995); and, the absence of structured activities (Bursik & Grasmick, 1993) have all been related to increased involvement of youth in violence (Brewer, Hawkins, Catalano, & Neckermen, 1995). The eventual scoring of this item will consider indicators of the overall quality of the neighborhood and community, such as the availability of drugs and firearms in the neighborhood, neighborhood adults involved in crime, gang activity, and an absence or lack of prosocial, structured, and supervised activities.Social stressors impair effectiveness of coping mechanismsLife stress one of highest correlates of imminent aggression (Thornberry)Social supports improve coping mechanismsSocial support is not the same as parental involvement (which can sometimes be pathologically “supportive”)
20 Protective Factors or Strengths Decreases the potential harmful effect of a risk factor (a ‘buffer’)Pro-social activities/sportsPositive social supportExcelling at school
21 Elements of a Comprehensive Risk Assessment Elements of a Comprehensive Risk Assessment. Most Appropriate for ‘Deep End’ UseEvidence-Based AssessmentStatic Risk FactorsDynamic Risk Factors (criminogenic needs)Protective or Responsivity FactorsWell-Being or Non-Criminogenic NeedsNote – this is in contrast to a “Brief Risk Assessment’Risk: likelihood of future offendingRisk factor: anything that increases the probability that a person will re-offend:Static Risk Factors – do not changeDynamic Risk Factors (criminogenic needs) – changeable, targets for services and interventionProtective factor or strength: decreases the potential harmful effect of a risk factorResponsivity factor: characteristics of the individual that can affect intervention success≠Enables reassessment of risk level to measure change
22 How to Pick an Evidence-Based Risk Assessment Tool – 5 elements (Vincent et al., 2009) Purports to assess “risk” for re-offendingHas a test manualWas developed for, or validated on, juvenile justice youth in the right setting (gender, race, etc)Demonstrates reliability - two independent raters would reach similar conclusionsDemonstrates a strong relation to re-offending (research refers to this as predictive validity)Preference should go to those that permit reassessment if it is being used for case planningADD MORE CRITERIA FOR PV AND RELThe most important point on this slide is that At least 1 peer-reviewed study has demonstrated a strong relation to future violence and/or recidivism by an independent party who has no stake in the sale of the instrument.Suggest that with regard to risk assessment tools, any future definition of EBP would undoubtedly include the need for research evidence (typically more than one study) that the thing that the instrument is trying to predict actually is predicted with some degree of consistency, within the context and population for which the instrument was developed.
24 Eight Steps of Implementation Getting readyEstablishing stakeholder and staff buy-inSelect the tool and prepare to use itDeveloping policies and other essential documentsTrainingPilot test implementationFull implementationTasks to promote sustainabilityNote #2 – convening a stakeholder group is key – discuss who should be part of that groupCase examples
25 Point in the Process for Using Risk Assessment: Pre-Adjudication Family ServiceDivert?Risk AssessmentSubstance Abuse TreatmentReduce Re-Arrest?FormallyProcessPeer RelationsReduce re-arrest for high risk offendersDisposition
26 Substance Abuse treatment Point in the Process for Using Risk Assessment: Post-Adjudication/Pre-DispositionFamily ServiceProbationSubstance Abuse treatmentRisk AssessmentGroup HomeReduce Re-Arrest?Mental HealthSecureLife Skills
27 What Else Should Be In Place for An Effective Risk Assessment ‘System’? Policies about…Use of the risk assessment toolRequirements for staff trainingAdministration (when, by whom, to whom)How the information will be communicated (to courts, treatment providers, etc.) & appropriate information-sharing agreementsQuality assurance procedures (by supervisors, via data reports)Underscore importance of “protecting” the information gathered so it cannot be used against a youth.
28 What Else Should Be in Place? cont. Policies and documents structuring and ‘standardizing’ how the information will be used in decisions….Disposition recommendationsCase planService referrals (application of a Service Matrix)Supervision level
29 Service Matrix Example (partial) YLS Risk/Need AreaSubstance AbuseFamily Circumstances/ParentingEducation/EmploymentLow riskFill in servicesModerateHigh risk
30 What can Risk assessment if implemented properly do for you? The next set of slides show some results from the Risk Assessment in Juvenile Probation: Implementation Study. This was a pre-post study of the impact of implementing an evidence-based risk assessment tool (the YLS/CMI or the SAVRY) in six probation offices. A sample of adjudicated youth was selected before a tool was implemented (pre) and compared to a sample of youth who were adjudicated after a tool was implemented (post) in each office. The samples of youth varied from 100 to 250 depending on the probation office. Youth were tracked for up to 13 months to record their disposition, any placements, services received, and their levels of supervision.
31 Results of the MacArthur Risk Assessment Study: Reducing Use of Out-of-Home Placements (Vincent et al., 2012)OR = 0.56OR = 0.37This demonstrates post-adjudication placement rates before and after a risk assessment tool (e.g., the SAVRY) was put in place in one probation office. The size of the pre-SAVRY sample was 205 adjudicated youth the year prior to SAVRY implementation. The post-SAVRY sample was a different 205 youth who had been adjudicated the year following the SAVRY implementation, which was matched to the pre-SAVRY sample along a number of youth characteristics (eg., race, age, gender, severity of offense, prior arrest history). The youths who were placed AFTER the SAVRY was implemented were more likely to be high risk than moderate or low risk.Placement was defined as any out-of-home removal – including treatment/inpatient, detention, residential homes, and secure correctional facilities. It does NOT include foster care placements.The slide separates their initial placement (meaning their first location following adjudication was a placement) from any placement during the study period up to 13 months on probation. The purpose was to separate placements likely to be given at disposition from placements likely to be given for a later revocation of probation or new offense. Placements in BOTH detention AND corrections decreased. This was the pattern in the two probation offices tested in this study that started with initially high placement rates.Analyses involved propensity score matching (Matching nearest neighbor w/common support) and regression analyses:Groups remained significant after including the following co-variates: # past violent offenses, # of prior charges,Any placement = chi-square(4) = 23.52, p < .000; rtestgroups (21), p = .006 Exp(B) = .56Initial placement = chi-square(3) = 30.43, p <.000; rtestgroups -.98(.25), p < .000 ExpB(.37)TIME IN PLACEMENT – the average amount of time spent in placements (among youth who had a placement) was not significantly different between these groups – after MATCHING And accounting for covariates (nonviolent felony & number of priors). Using ANCOVA F(1, 158) = .029, p - .86:
32 Improved Allocation of Services (Vincent et al., 2012) Mean # ServicesThe number of services referred for youth on probation was strongly related to the youths’ risk level on the SAVRY after the SAVRY was implemented. The number of services completed by these youth was also significantly related to risk level. This association between youths’ risk level and the number of services assigned by probation officers was evident in four out of the six probation offices that implemented a risk assessment – be it the SAVRY or the YLS
33 Reducing Use of Maximum Levels of Supervision on Probation (Vincent et al., 2012) % At Supervision LevelThis demonstrates post-adjudication rates of individuals on different supervision levels rates before and after a risk assessment tool (e.g., the SAVRY) was put in place in one probation office. The size of the pre-SAVRY sample was 205 adjudicated youth the year prior to SAVRY implementation. The post-SAVRY sample was a different 205 youth who had been adjudicated the year following the SAVRY implementation, which was matched to the pre-SAVRY sample along a number of youth characteristics (eg., race, age, gender, severity of offense, prior arrest history). The youths placed on higher levels of supervision were almost always the higher risk youths.The slide represents everyone’s first assigned supervision level when initially starting probation. The post-SAVRY’s rates of use of maximum and intensive supervision levels were significantly lower than the Pre-SAVRY group. And, post-SAVRY there was not a bump of youth into the higher levels of supervision after being on probation after a while.
34 In Some Cases, Reduce Recidivism OR = .48OR = .47OR = .42Northamptom –Used nearest neighbor w/common support matching and included covariates in analyses. Final total N = 442 (C=221; T=221)Ave follow-ups (minus those removed for no release) Comp sample = months; Test sample = months______________/**** Northampton- these covariates were related to recidivism. ---don't use - these variables all have missing data/*** pT2viol=Nothing/***pT2any=rtestgroups, RR_schoolattend/***pT2nonviol=RR_schoolattend/***pT2sex=Nothing/***pt2violation=RR_schoolattend/*pviolentrecid=testgroups/* panyrecid=testgroups, RR_schoolattend/* pnonviolrecid=testgroups/* psexrecid=NOTHING/* pviolationsrecid=RR_schoolattendCox regressions – time to offending: Betas for rtestgroups in final model - noViolence rtestgroups– B = -.75; SE=.37; Wald=4.16; p=.04; Exp(B) = .47Nonviolent rtestgroups– B = -..74; SE=.23; Wald=10.14; p< .001; Exp(B) =.48probation violations rtestgroups - B = -.87; SE=.37; Wald = 5.38; p = .02; Exp(B) = .42Any recid rtestgroups– B = -.84; SE=.20; Wald = 17.93; p< .001; Exp(B)=.43New Petitions – 18 month follow-up (N = 442)
35 What Risk Assessments Do NOT Do NOT prescriptiveThese types of general risk assessments are NOT appropriate for identifying risk for sexual offendingNOT mental health assessmentsThey also do not identify potential mental health problems in need of an assessmentTypically do NOT include items that are unrelated to future offending, like “well-being needs” (e.g., special education, depression, trauma)POINT is that no valid tool has been created for ALL purposes. Different tools are for different things.– service delivery and decision-making is part of the policy, which should be customized for each jurisdiction– requires special sexual risk tools
36 Summary: Benefits of Risk Assessment Helps states to conserve resourcesCan help improve outcomes for young offenders…..Improved chance of reducing risk = reduction in re- offendingBetter use of placement and monitoring = improved functioning and cost-savingsProvides a means for data tracking to evaluate….Changes in youths’ risk (if using a dynamic assessment)Services and decisions pertaining to out-of-home placementCaveat: The benefits are unlikely to be attained without appropriate implementationNote that when a tool isn’t implemented well it is unlikely to lead to any change in practice. It is essential that there be a policy and staff training about how the tool will be used in decisions. It is also essential to have buy-in from key stakeholders (judge, attorneys, etc) in order for this to work.
37 Questions?NYSAP website: - for downloading the manual