Centre for Research on Children and Families

National Outcomes Matrix: Evidence of Intervention Effectiveness Nico Trocmé with Tony Esposito, Martin Chabot & Lorry Coughlin Theory & Wisdom: Promoting Evidence Based Decision Making Edmonton, AB , May 28th Centre for Research on Children and Families

Management Practice An Incremental Multi-Level Outcomes Approach Financial Based Management Client Tracking System Using Proxy Measures Management Integrated Outcomes Tracking System An Incremental Multi-Level Outcomes Approach Practice Outcome Based Case-Planning + Measures for Clinical Practice Reactive Practice Centre for Research on Children and Families

Selecting indicators: Can indicators be clearly linked to seminal child welfare service objectives? Are they easily understood? Can they be measured using existing information systems? Centre for Research on Children and Families

Competing objectives of child welfare Child Protection Child Well-Being Family and Community Support Centre for Research on Children and Families

NOM & Pilot Test Indicators Family & Community Support Permanence Well-Being Safety 1.Recurrence & 2.Injury 3.School delay & 4.YCJA 8. Court 9. Housing stability & 10. Community-based placements 5.Placement 6. Moves in care & 7.Time in care Centre for Research on Children and Families

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 7

Recurrence cohort: All closed Cases served, tracked for 12 months Mean Age 2002 580 8.1 2003 495 8.0 2004 575 7.8 2005 530 8.3 2006 522 8.4 2007 395 8.2 6 year average SDC AM AM / IT IT SDNC Nico Recurrence cohort: All closed Cases served, tracked for 12 months 8

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 9

Recurrence by reason for report Total Closed cases 373 44 12 92 8 529 Non recurring 322 40 7 72 449 Recurring 51 4 5 20 80 Recurrence rate 13% 9% -- 21% 15%

Recurrence by age at service start Methodological notes: Maintenant, meme enfant peut etre dans plusieurs cohortes, 17 ans et moins a fin de AM Recurrence of maltreatment tracks children who had at least one substantiated recurrent event of maltreatment within 12 months of their file closing. Retained recurrence tracks maltreatment for children while receiving services. (Cohort 2002-2005). For example: approximately 24% of children receiving AM services had a recurring event of maltreatment while receiving AM services. RRAM Numerator () = number of AM closed with at least one retained recurrent event while the child is receiving AM services. RRAM Denominator () = number of AM closed. RRAM Total = 23,8 %. The NOM preliminary measures suggest that 22% of children who had ongoing child welfare involvement that ended in the referring year had a recurring event of maltreatment within that year. Total Closed cases 82 102 134 122 89 529 Non recurring 75 118 98 69 449 Recurring 7 13 16 24 20 80 Recurrence rate 8% 12% 11% 19% 22% 15% 11

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 12

School Delay Rate Proportion of agency children who are at least 1 year behind age appropriate grade by placement type (all years combined) Methodological notes: Maintenant, meme enfant peut etre dans plusieurs cohortes, 17 ans et moins a fin de AM Recurrence of maltreatment tracks children who had at least one substantiated recurrent event of maltreatment within 12 months of their file closing. Retained recurrence tracks maltreatment for children while receiving services. (Cohort 2002-2005). For example: approximately 24% of children receiving AM services had a recurring event of maltreatment while receiving AM services. RRAM Numerator () = number of AM closed with at least one retained recurrent event while the child is receiving AM services. RRAM Denominator () = number of AM closed. RRAM Total = 23,8 %. The NOM preliminary measures suggest that 22% of children who had ongoing child welfare involvement that ended in the referring year had a recurring event of maltreatment within that year. 7 to 17 years of age TOTAL (N) 63% 26% 24% 5000 13

School delay by age & sex: All years P=0.000

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 15

Service cohort: all retained cases, up to 36 months after report Mean Age 2002 950 7.8 2003 870 2004 750 7.6 2005 890 7.7 4 year average SDC AM IT Nico Service cohort: all retained cases, up to 36 months after report SDNC Centre for Research on Children and Families 16

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 17

Youth Criminal Justice involvement by reason for the report Total Children in cohort 73 34 7 67 3 184 No YCJA 62 31 49 152 YCJA 11 18 32 YCJA Rate 15% 8% *--% 26% 17% * Too few cases to calculate a meaningful rate 18

Youth Criminal Justice involvement by age Total Methodological notes: Maintenant, meme enfant peut etre dans plusieurs cohortes, 17 ans et moins a fin de AM Recurrence of maltreatment tracks children who had at least one substantiated recurrent event of maltreatment within 12 months of their file closing. Retained recurrence tracks maltreatment for children while receiving services. (Cohort 2002-2005). For example: approximately 24% of children receiving AM services had a recurring event of maltreatment while receiving AM services. RRAM Numerator () = number of AM closed with at least one retained recurrent event while the child is receiving AM services. RRAM Denominator () = number of AM closed. RRAM Total = 23,8 %. The NOM preliminary measures suggest that 22% of children who had ongoing child welfare involvement that ended in the referring year had a recurring event of maltreatment within that year. Children in cohort 57 48 79 184 No YCJA 50 37 65 152 YCJA 7 11 14 32 YCJA Rate 12% 22% 17% Centre for Research on Children and Families 19

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 20

Placement rate by reason for the report Total Children in cohort 524 110 29 92 6 761 No placement 407 95 26 49 3 580 Placement 117 15 43 181 Placement Rate 22% 13% --% 46% 23%

Placement rate by age Total 117 170 190 205 79 761 77 150 162 149 42 Methodological notes: Maintenant, meme enfant peut etre dans plusieurs cohortes, 17 ans et moins a fin de AM Recurrence of maltreatment tracks children who had at least one substantiated recurrent event of maltreatment within 12 months of their file closing. Retained recurrence tracks maltreatment for children while receiving services. (Cohort 2002-2005). For example: approximately 24% of children receiving AM services had a recurring event of maltreatment while receiving AM services. RRAM Numerator () = number of AM closed with at least one retained recurrent event while the child is receiving AM services. RRAM Denominator () = number of AM closed. RRAM Total = 23,8 %. The NOM preliminary measures suggest that 22% of children who had ongoing child welfare involvement that ended in the referring year had a recurring event of maltreatment within that year. Children 117 170 190 205 79 761 No Placement 77 150 162 149 42 580 Placement 40 20 28 56 37 181 Placement Rate 34% 11% 14% 27% 46% 23% Centre for Research on Children and Families 22

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 23

Placement Cohort AM SDC IT SDNC Mean Age 2002 330 10.4 2003 325 11.6 2004 294 10.5 2005 320 11.5 4 year average 11.1 Placement cohort: all placed children, up to 36 months after initial placement Nico AM SDC IT SDNC Centre for Research on Children and Families 24

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 25

Moves in Care Number of moves (# of children) 1 2-3 4+ Total children 28% 37% 19% 1 21% 20% 25% 2-3 23% 30% 4+ 26% Total children 332 100% 322 289 Total Moves 767 641 723 Average moves 2.5 2.1 2.4 26

Moves in Care: by reason for report Total Number of moves 22% 13% 25% 14% 54 19% 1 27% 29% 23% 44% 76 26% 2-3 30% 20% 31% 85 4+ 21 38% 32% 11% 74 Total children 155 100% 24 4 97 9 289 Total Moves 338 68 8 293 16 Average moves 2.2 2.8 2.1 3.1 1.7 27

Moves in Care by age at entry Total Number of moves 32% 19% 17% 11% 20% 54 1 16% 47% 39% 21% 76 26% 2-3 44% 25% 27% 31% 85 30% 4+ 8% 9% 37% 33% 74 Total children 38 100% 32 41 67 111 289 Total Moves 58 47 72 227 319 Average moves 1.5 1.4 1.6 3.5 2.8 28

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 29

Time in Care by age at entry Outcome after 36 months percentage by outcome (median in days) Total Children %(N) Return Home 41% (225) 34% (27) 47%(58) 41% (66) 65%(106) 52%(167) Adoption 10% (502) 1%(3) Other 28% (437) 26% (558) 14%(29) 22% (107) 27%%(250) 24%(77) Still in Care 21% (1005) 40% (1008) 39%(999) 37% (1105) 8%(1017) 23%(1022) Total Children 29 35 42 71 145 322 30 30

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 31

Court Involvement by reason for report Total Children in cohort 524 110 29 92 6 761 No Court Involvement 295 86 22 47 2 452 Court Involvement 229 24 7 45 4 309 Court Rate 43% 21% --* 48% 40% * Too few cases to calculate a meaningful rate

NOM Dashboard: Pilot Test 02-08 02-03 03-04 04-05 05-06 06-07 07-08 Recurrence Cases served re-substantiated within 12 months of closing 15% 16% 13% 9% School Delay Agency children who are at least 1 year behind age appropriate grade compared to norm 24% 5% 32% 6% 30% 27% 4% 25% Youth Criminal Justice Rate Investigated children (12 to 15 years old) who experienced youth criminal justice services within 36 months of initial investigation 22% 17% 12% NA Placement Rate Investigated children who experience at least one placement (>3 days) within 36 months of initial investigation 21% 23% Moves in care Placement changes (>3 days) or reunification breakdowns within 36 months of initial placement 2.5 2.1 2.4 1.90 Time in care (% current status / median days per current status) Return home Adoption Other Still in care 260 56% / 107 2% / 850 19% / 183 22% /1025 255 52% / 70 1% / 502 24% / 224 23% /1011 385 55% / 201 3% / 805 16% / 313 26% /1095 Court Involvement Children who experience at least one court event within 36 months of initial investigation 40% 42% 33% 33

For more information: www.cecw-cepb.ca nico.trocme@mcgill.ca tonino.esposito@mail.mcgill.ca www.cecw-cepb.ca Centre for Research on Children and Families

www.cecw-cepb.ca Centre for Research on Children and Families