Introductory Points Introductions. Introductions. Assessment V Evaluation Assessment V Evaluation ‘Blunt’ Tools ‘Blunt’ Tools –What is a positive outcome? –Outcomes for individuals V outcomes for groups. –Tool ‘efficacy’ (moods, cultural variables, questioning techniques) –Frequently ‘deficit’ mindset. Experiences from education e.g. Neale Literacy ( Accuracy / Comprehension / Rate), ‘best’ schools.

The Boys’ Town Model

8 Planks (The Roots) 1. Presence 2. Preventive 3. Plan 4. Professional 5. Positive 6. Personal 7. Parental involvement 8. Play

Wraparound elements The Tree Trunk Education and schooling support Education and schooling support Behaviour management programs that ensure safety and sustainability of placements and school positions Behaviour management programs that ensure safety and sustainability of placements and school positions Life and Social Skill development programs Life and Social Skill development programs Individual and family counseling and psychological services Individual and family counseling and psychological services Family restoration Family restoration Respite Respite

Theoretical Framework The Ecology of the Child (The branches) The Ecology of the Child (The branches) –Home –School –Peers –Community Circles of Courage (Larry Brendtro Circles of Courage (Larry Brendtro (The Fruit) (The Fruit) –Mastery (Competence) –Belonging(Significance) –Independence (Power) –Generosity (Virtue)

Some general points –Children disengaged (75% non school attendees) –Most with Mental Health diagnoses –Voluntary nature of program. –Inappropriate behaviour –Family Preservation

EVALUATION FRAMEWORK; EVIDENCE OF SUCCESS QUANTITYQUALITY INPUT EFFORT How Much did we do? How well do we do it? OUTPUTEFFECT Is Anyone better off? Quantity Quality Mark Friedman; Trying Hard Is Not Good Enough

Assessment Family Assessment Purposes Family Assessment Purposes –Client meets entry / exclusion criteria –Information gathering. (history of interventions, diagnosis, family dynamics) Other Assessment tools Other Assessment tools - Achenbach - Achenbach - Literacy numeracy - Literacy numeracy - Resiliency - Resiliency Assessment Tools in Evaluation Assessment Tools in Evaluation

Tools Family Assessment to establish goals Family Assessment to establish goals Achenbach Achenbach Modified Family Coping Scales Modified Family Coping Scales Resiliency Scales Resiliency Scales Literacy and numeracy tests Literacy and numeracy tests

EVALUATION How Much Service? 104 Enquiries 104 Enquiries 79 families at family talk 79 families at family talk 52 families in program 52 families in program Average stay 7 months Average stay 7 months 39 back in school 39 back in school 34 Risk of Harm Notifications 34 Risk of Harm Notifications 69 Re-entry meetings 69 Re-entry meetings 14 camps 14 camps

EVALUATION How well do we do it? Family Evaluations Top points Family Evaluations Top points –Goals established on entry (9.8 av) –Regular Counselling (9.7) –Family’s own participation in program (9.7) –(least) Re-entry processes (7.7) Participants Evaluation Top Points Participants Evaluation Top Points –Clear behaviour consequences (8.7) –Assistance with transition (8.7) –Family’s own involvement in program (8.6) –(Least) Small groups (6.3)

EVALUATION Is Anyone Better Off? Average attendance exceeding 90% at Boys’ Town Average attendance exceeding 90% at Boys’ Town 80% in full time school, work or TAFE after 2 years 80% in full time school, work or TAFE after 2 years In 6 months improvement average of more than 2 years in comprehension, and more 4 years in reading accuracy. In 6 months improvement average of more than 2 years in comprehension, and more 4 years in reading accuracy. Average gain in spelling = 2 years in 6 months Average gain in spelling = 2 years in 6 months Family Comments overwhelmingly positive Family Comments overwhelmingly positive

Is Anyone Better Off? Improvements on Achenbach for Students in Clinical/Borderline range Anxious50% Withdrawn/Anxious78% Somatic Complaints56% Social problems63% Thought problems38% Attention Problems67% Rule Breaking74% Aggressive behaviour52% NON ClinicalAverage 10% Externalised 8% Internalised improvements

New Resiliency Tool MASTERY Optimism Median lowest 10% MASTERY Optimism Median lowest 10% Self Belief in problem solving bottom 15% Adaptability and flexibility bottom 15% SENSE OF RELATEDNESS SENSE OF RELATEDNESS Trust well below average Belief in support system well below average Belief in ability to make friends well below average

New Resiliency Tool EMOTIONAL REACTIVITY EMOTIONAL REACTIVITY Belief that their emotional arousal hinders their daily functioning (top 15% of population) BUT BUT Participants normalise their own reactions and their ability to recover.

Challenges Tools that measure what you want. Tools that measure what you want. Easy to administer. Easy to administer. ‘less blunt’ tools. ‘less blunt’ tools. Making sure information gained is used to inform case plan and improve program outcomes. Making sure information gained is used to inform case plan and improve program outcomes.