Presentation is loading. Please wait.

Presentation is loading. Please wait.

Prevention and Early Intervention Pilot program Client Outcomes Studies Australasian Evaluation Society Lunchtime Seminar February 2007 Presented by Dr.

Similar presentations


Presentation on theme: "Prevention and Early Intervention Pilot program Client Outcomes Studies Australasian Evaluation Society Lunchtime Seminar February 2007 Presented by Dr."— Presentation transcript:

1 Prevention and Early Intervention Pilot program Client Outcomes Studies Australasian Evaluation Society Lunchtime Seminar February 2007 Presented by Dr Terrence O’Brien

2 Evaluation Evaluation should not be seen as an end product or one off event. Evaluation is a participatory process. Evaluation is an essential tool for continuous quality improvement.

3 Client Outcomes Evaluation Application to human services is challenging Changes in family’s behaviours due to interventions needs to be measured, but not easy The question is more about how rather than if it can be done or not..

4 What is an Outcome evaluation? Outcome evaluations/studies look to the extent to which program participants experience benefits or changes in their behaviour and life circumstance and indications that these changes are related to the services received. (Mikka 2001)

5 Client Outcomes Evaluation Previous difficulty in evaluating clients’ outcomes. Outcomes in humans services hard to define and measure. Sensitivity to professional and client needs. Human Services not conducive to rigorous experimental approaches. Research in this area is becoming more popular. Growing trend of outcome measures being used to monitor funded NGOs.

6 Outcomes Evaluation Help to inform the choice and validity of intervention, ensuring best/evidence based practice principles are followed Allow the organisations to be outcome focussed. Strengthen existing services. Target effective services for expansion. Identify staff and volunteer training needs. Develop and justify budgets. Prepare long-range plans. Focus board members' attention on program issues.

7 PEI Pilot Programs Client Outcomes Studies Putting Families First Future Directions Initiatives – PEI 1,2,4, 5, 7 and 8 PEI and Indigenous Family Support Trials and Pilots Anecdotal evidence of positive outcomes from trials (one year) Client Outcome Studies needed to be developed for the pilot evaluation (3 years)

8 PEI Pilot Programs 25 Trial Programs 18 Pilot Programs 12 Communities and 6 Child Safety Evaluation Processes Demographic and Case Management and PEI Principles Survey Client Outcomes Studies

9 MISSION AUSTRALIA-PROJECT CIRCUIT BREAKER OUTCOMES EVALUATION PROJECT CONCEPT When my service does (our services do) - intervention Strengths based early intervention and prevention- With children, parents, families who- Consist of families with children between the ages of 10 and 16 years who live in the Northern Suburbs of the Department of Communities Brisbane City region. When the context is- Referrals of families with students in secondary school who are experiencing relationship difficulties and other issues such as truanting and anti- social/deviant behaviour. What change is there in (what change am I wanting to see) – Stability and quality of family relationships and communication

10 Project concept cont. And how does this impact on the child by improving their – Level of positive engagement with education In terms of: - School attendance Behaviour at school

11 OUTCOMES EVALUATION PROJECT DESIGN Organisation: Mission Australia - Project Circuit Breaker 1. What are your Questions? When my service provides strength based, solution focused case management, is there improvement in the young person’s: school attendance, behaviour at school and behaviour at home during and after the intervention. Are there improvements in the quality of the family relationships during and after the interventions? What are the individual outcomes that can be attributed to each intervention strategy that is implemented with a family? 2. How will you find out? Methodology Type: Descriptive (Survey Design)/pre and post Interventions and after 3 months. 4 case studies. (Using the NCFAS Child Behaviour Checklist and other questionnaires devised by Project Circuit Breaker staff) Over what time: 3-6 month intervention periods over a total of 2 years

12 3. Sample Type of sample: Purposive sample - Case Study sample – consenting families How many families in the sample: 30 + Dependent on number of referrals received Description of the sample: Families and young people referred from Secondary Schools within Project Circuit Breaker’s catchment area. Families experiencing difficulties with young person’s behaviour. Young person experiencing difficulties with relationships parent/s and at school. 4. What will you do about: Confidentiality and privacy issues: Information about the purpose, duration and the role that families play in the study are given to families prior to request of information. Families will not be identified; codes will be attributed to each member of family. Informed consent: Forms to be signed Ethical issues: Using AES ethical principles 5. Tools and Techniques Types: Standardised tests – Pact c, NICFAS Quality of Family interactions, Child Behaviour Checklist for ages 6 –18 (home/school/self report) attendance records (3 months prior/3 months during intervention) and questionnaires devised by PCB staff, GO, and Parent/young person. Case Studies developed from case notes – semi structured interview at 3 months after intervention. Key characteristics to measure: Changes in presenting issues, Family interactions, Changes in behaviour at school, School attendance. When will you use them? : Pre and post intervention and 3 months after intervention

13 OrganisationOutcomes Sisters Inside - BOWS 1.Accommodation 2.General well being 3.Child reunification Centacare - RAIFSS 1.Engagement using culturally appropriate ways 2.Parenting skill and knowledge 3.Community strength Durumbal – Eagle Hawk Family Support Service 1.Improvements in family resilience 2.Self esteem 3.Improved budgeting and household management skills 4.Increased community engagements 5.Contact with support networks Client Outcomes

14 Mission Australia - Project Circuit Breaker 1.School attendance 2.Behaviour at school 3.Behaviour at home 4.Family Relationships 5.What are the individual outcomes that can be attributed to each intervention strategy that is implemented with a family? Lutheran Family Support Services - REFS 1.Young person’s behaviour 2.Parenting skills 3.Family relationships Mununjali Jymbi Centre 1. Family and community resilience and well being 2.Sustainable self determining family and cultural support systems and structures 3.Reduce the levels of representation of Aboriginal people and Torres Strait Islander people in statutory systems and crisis services Client Outcomes

15 Ozcare - GIFS 1.In the well being, school behaviour and attendance of children 2.Relationships and parenting skills of families 3.Coping skills of children with a disability Brisbane Youth Service – Parents Yes 1.Relationships 2.Parenting skill and knowledge 3.Connections to support networks 4.Participation in life Red Cross 1.Children’s school attendance and behaviour 2.Educational achievement Client Outcomes

16 Save the Children 1.Creative strategies of intervention 2.How does this assist the family to become involved in a therapeutic relationship with an FSW? 3.A child's Health, Emotional State, Connectedness and Intellectual state The Spot – The Spot for Kids 1.Family relationships 2.Contact with Spot and local community support networks 3. Child/children’s general behaviour at home, school and at The Spot Cairns Women’s Centre - OSP 1.Does the program help improve family’s social connectedness? 2.Is there more stability in the family’s housing situation? 3.Is there improved contact with community support networks? Client Outcomes

17 OrganisationInterventionClient SampleOutcome or Questions Will the organisations see improvements in Data collection techniques Data collection time frame Lutheran - REFS Family therapy, in home support & specialised group work Young persons aged 11- 16 and their families displaying difficult behaviours Young persons behaviour Parenting skills Family relationships Surveys and questionnaires devised by REFS and REU Pre, exit and 3 mths post Mission Australia - Project Circuit Breaker Strength Based intervention. Young persons aged 11- 16 displaying and their families difficult behaviours School attendance, Behaviour at school Behaviour at home Family Relationships What are the individual outcomes that can be attributed to each intervention strategy that is implemented with a family? Pact c, NICFAS Quality of Family interactions, Child Behaviour Checklist for ages 6 –18 (home/school/s elf report) Attendance records Questionnaires 3 months prior/3 months during interventi on and Survey design OrganisationInterventionClient SampleOutcome or Questions Will the organisations see improvements in Data collection techniques Data collection time frame Lutheran Family Support - REFS Family therapy, in home support & specialised group work Young persons aged 11- 16 and their families displaying difficult behaviours Young persons behaviour Parenting skills Family relationships Surveys and questionnaires devised by REFS and EU Pre, exit and 3 mths post Mission Australia - Project Circuit Breaker Strength Based intervention. Young persons aged 11- 16 displaying and their families difficult behaviours School attendance, Behaviour at school Behaviour at home Family Relationships What are the individual outcomes that can be attributed to each intervention strategy that is implemented with a family? Pact c, NCFAS Quality of Family interactions, Child Behaviour Checklist for ages 6 –18 (home/school/s elf report) Attendance records Questionnaires 3 months prior/3 months during interventi on and

18 Red CrossIn home and at school mentoring service. Primary school aged children and their families where there area indicators of abuse or neglect or not coping with parenting responsibilities Children’s school attendance and behaviour educational achievement Surveys and questionnaires devised by Red Cross. Pre, exit and 3 mths post OrganisationInterventionClient SampleOutcome or Questions Will the organisations see improvements in Data collection techniques Data collection time frame Survey design

19 OrganisationInterventionClient SampleOutcome or Questions Will the organisations see improvement in Data collection techniques Data collection time frame Save the Children Practical assistance and counselling. High support need families residing in caravan parks Creative strategies of intervention How does this assist the family to become involved in a therapeutic relationship with an FSW. A child’sHealth, Emotional State, Connectedness and Intellectual state Session activity Data Qualitative research Action research minutes 6 mths of data The Spot – The Spot for Kids Family based training personal development program Families with children up to the age of 12 Family relationships Contact with Spot and local community support networks Child/children’s general behaviour at home, school and at the spot Case notes, NCFAS, EU data Pre and Post Case Studies

20 OrganisationInterventionClient SampleOutcome or Questions Will the organisations see improvement in Data collection techniques Data collection time frame Sisters Inside - BOWS Parenting support for reconnecting mothers and their children. Mothers who are leaving prison Accommodation General well being Child reunification Forms with rating scales, questionnair es, case notes Pre and Post Ozcare - GIFS Collaborat- ive Case work Young children, families and children with a disability In the well being, school behaviour and attendance of children Relationships and parenting skills of families and coping skills of children with a disability Rating Scales, case studies, diaries and Observation s Pre and Post Case Studies

21 Brisbane Youth Service – Parents Yes Parenting support program Parents under the age of 25 Relationships Parenting skill and knowledge Connections to support networks participation in life Goal attainment Scale, case notes, Eco Map, Family Strengths FDI 2 Database 3 mths of data OrganisationInterventionClient SampleOutcome or Questions Will the organisations see improvement in Data collection techniques Data collecti on time frame Case Studies

22 OrganisationServiceClient Sample Outcome or Questions Will the organisations see improvement in Data collection techniques Data collection time frame Centacare - RAIFSS Fortnightly face to face integrated family support where Children and young people who are at risk of statutory intervention Engagement using culturally appropriate ways Parenting skill and knowledge Community strength Narrative Inquiry Case Notes Observation Inter- vention period Durumbal – Eagle Hawk Family Support Service Youth connecting with community activities. 6 Aboriginal and Torres Strait Islander families with young people aged 12 – 19 years." Improvements in family resilience Self esteem Improved budgeting and household management skills Increased community engagements Contact with support networks Narrative Inquiry Inter- vention period Narrative Inquiry

23 Mununjali Jymbi Centre 1.Family and Cultural Community Development Support3. Healing & Intervention Programs Aboriginal and Torres strait Islander families. Family and community resilience and well being Sustainable self determining family and cultural support systems and structures Reduce the levels of representation of Aboriginal people and Torres Strait Islander people in statutory systems and crisis services Narrative Inquiry Action Research Inter- vention period OrganisationServiceClient Sample Outcome or Questions Will the organisations see improvement in Data collection techniques Data collection time frame Narrative Inquiry

24 Parents YES Outcomes Study –relationships; –parenting skill and knowledge; –connections to support networks; and –participation in life. Improvements in positive child/parent relationships and interactions Children are reaching their development milestones Parenting skills and knowledge has increased Connections to support networks are strong Changes have been seen in life participation An increase in stable, safe, secure and adequate housing

25 Project Circuit Breaker’s, Client Outcomes Study a young person’s attendance and academic performance at school; general changes in the young person’s behaviour; and movement towards improvement in family functioning –all families perceived there was significant change to the presenting issues due to intervention; –they reported the influence of intervention on goal achievement; –they perceived that progress towards goals was due to intervention; –all young people maintained or showed improved grades in at least 3 out of 4 school subjects after intervention, yet their school attendance showed little change between initial and closure measurements; and –all young people scored lower (behaviour improvement) after intervention on at least one of the oriented scales scores (Child Behaviour Checklist and the Youth Self Report).

26 Red Cross’s Intensive Family Support program Client Outcomes Study the children’s school attendance and behaviour; and educational achievement. –improvements in school attendance and positive improvements in behavior for children both at home and at school; –positive feedback from the children’s teachers regarding educational achievement and attitude; and –most families made perceived and made positive progress towards their goals.

27 The Client Outcomes study for the Munujali Jymbi Centre improvement in family and community resilience and well being (abbreviated); establishment of sustainable self determining family and cultural support systems and structures reduction in the levels of representation of Aboriginal people and Torres Strait Islander people in statutory systems and crisis services. That there is increased community acceptance of the Centre and a high level of involvement in their programs; 5 local people have been trained and are now employed by the centre; The service has provided mentors for young people, children and their families; and The community reports that they have increased self confidence and direction and have a broader vision of the future.

28 The Eaglehawk Indigenous Family Support Program’s Client Outcomes family resilience and self-esteem; household budgeting and management skills; and community engagement and contact with support networks. both families exhibited improvements in family resilience; the social and emotional wellbeing of the families improved; a direct correlation between budgeting / household management and secure housing / food supply, with both families showing improvement in each; external relationships and health management improved. Improvements in positive child/parent relationships and interactions


Download ppt "Prevention and Early Intervention Pilot program Client Outcomes Studies Australasian Evaluation Society Lunchtime Seminar February 2007 Presented by Dr."

Similar presentations


Ads by Google