Using mixed methods to investigate effectiveness and costs of key worker services for disabled children Tricia Sloper, Social Policy Research Unit, University of York Jeni Beecham, Personal Social Services Research Unit, University of Kent

Funders Treasury Evidence Based Policy Fund Department of Health Department for Education and Skills Welsh Assembly Government

Background: experiences of families with disabled children Difficulties in finding out about and accessing services Poor information provision Lack of coordination between services High levels of unmet need Policy recommendations re key workers since 1976

Evidence on provision of and effectiveness of key workers Effects of key workers: Better relationships with services Higher morale Less isolation and feelings of burden Improvements in receipt of information, access to services and fewer unmet needs Partnership and family involvement Job satisfaction for staff Recent increase in numbers of key worker services Different models of service being implemented No information on the costs of these supports

Aims of the research To compare the implementation and operation of different models of key worker services. To assess the outcomes for parents and children of the provision of different models of key worker services. To investigate sources of funding and costs of different models of key worker services. To identify the features of the services that contribute to improved care for disabled children and their families. To inform standards of good practice in services for disabled children and their families.

Why mixed methods Information on existing key worker services – e.g. what agencies are involved, what model used, how many key workers, how many families, costs etc – quantitative data – survey. More detail on different models – case studies of selected services looking at: Experiences of managers and staff – qualitative Outcomes for families – quantitative Experiences of parents and children – qualitative Features of services related to better outcomes for families – quantitative – path analysis Features of families and services associated with costs – quantitative – analyses of cost variations

Methods 1. Survey of key worker/care coordination schemes in UK. 2. In seven services: Interviews with staff – key workers, service managers, steering group members (n=87); teachers and heads of schools (n=15) Questionnaires completed by parents (n=205) and children (n=30) who received the services Interviews with parents (n=68) and children (n=9)

How methods related to each other (1) Survey – basic information on characteristics and costs. Informed selection of case study services Interviews with staff – detail on the selected services, staff evaluations of important characteristics, advantages and disadvantages of their model. Informed selection of independent variables for analysis of outcomes, and interpretation of quantitative analysis. Informed estimation of the total and unit costs, and the analysis of variations in the contact costs.

How methods related to each other (2) Questionnaires to parents and children – quantitative data on use of the service and outcomes. Path analysis for each outcome measure Calculation of key worker-family contact costs Analysis of the variations in contact costs Interviews with parents and children – experiences of receiving the service. How do these fit with results of path analysis? And the cost analysis? Experiences informed our interpretation.

Survey key findings Differences in: Designated v non-designated key workers v both Management Funding Training for key workers Supervision of key workers Multi-agency involvement Length of time service in operation

Staff interviews Important issues for staff Advantages and disadvantages of different types of key workers Training Supervision Management Understanding of the key worker role - job description Sharing of information and emotional support Funding Time to do the job

The outcomes we measured Impact of key worker on parents quality of life Parent satisfaction with the service Parents unmet needs Childrens unmet needs

Path analysis A series of analyses based on assumptions of potential causal order reflected in the grouping of variables into blocks. Context: characteristics of services (survey & interviews with staff) and of families (parent questionnaires) Mechanism: what key worker does with the family Outcomes for families (parent questionnaires)

Blocks of variables for path analysis Block 1: family and service context Block 2: service mechanisms Block 3: process outcome Family outcomes Family and child context: -age of child -level of disability -social class Service context: -dedicated funding -length of time in operation -regular training, supervision and peer support -service manager -clear key worker job description -type of key worker -parent representation on steering group Aspects of key worker role Key worker contacts with family: -frequency -duration -appropriate -proactive Measure of Process of Care scores Impact on quality of life Unmet parent needs Unmet child needs Satisfaction with key worker service

Path analysis of impact on parental quality of life (QOL) BLOCK 1 Family and Service Context BLOCK 2 Service Mechanisms BLOCK 3 Service Process Outcome Family Outcomes Shorter length of time service in operation Regular training supervision and peer support Service manager and clear job description High aspects of key working score (0.37) (0.36) (0.55) Positive impact on QOL (0.18) = paths from variables with no significant bivariate association with outcome measure (0.55) = beta coefficient values

Path analysis of satisfaction with key worker service BLOCK 1 Family and Service Context BLOCK 3 Service Process Outcome Family Outcomes Regular training supervision and peer support Shorter length of time service in operation Dedicated funding BLOCK 2 Service Mechanisms Appropriate level of contact with key worker High aspects of key working scores Longer telephone contacts with key workers Higher child disability score Higher satisfaction with key worker service (0.36) (0.37) (0.23) (0.20) (0.13) (0.65) (0.14) = paths from variables with no significant bivariate association with outcome measure (0.50) = beta coefficient values

Path analysis of parent unmet need scores BLOCK 1 Family and Service Context BLOCK 3 Process Outcome Family Outcomes Older child Longer length of time service in operation No regular training supervision, peer support BLOCK 2 Service Mechanisms Low aspects of key working score Not enough contact with key worker Lower child disability score High parent unmet need (0.34) (0.36) Low measure of process of care scores (0.43) (0.17) (0.19) (0.51) (0.42) (0.20) = paths from variables with no significant bivariate association with outcome measure (0.55) = beta coefficient values

Cost estimation in the seven sites Managers interview Survey Publicly available costs data Total annual costs for each service Contact cost: face-to-face visits and telephone (n=159) Family interviews: frequency and duration of key worker contact Unit cost per average worker hour: £24 - £42 Staff interviews: Use of their time, 25-50% on face-to-face contact Costs of providing key worker support

Contact costs over three months 13 families had no contact 38 families: between £1 and £ families: between £50 and £ families: between £100 and £ families: between £200 and £ families: between £500 and £ families: greater than £1000. Mean cost (£150) is a useful figure – but it is logical that if family's characteristics, needs and circumstances are different, so too will be the support cost.

Contact costs over three months Excludes nine families with support costs of more than £500

Why contact costs might vary Contact costs Child and family characteristics and needs Parents outcomes (n=4) Parents attitudes to the service Key worker attributes Key worker activities Scheme characteristics

Why contact costs vary (bivariate) Child characteristics and needs Child continence difficulties (+) Child seen hospital doctor in previous 3 months (+)* Total disability score (+)* Parent outcomes Total quality of life score (+) Total satisfaction score (+)* Scheme characteristics Dedicated funding (+)* Parent rep. Steering Group (-)* * Results remaining valid when adjusted for scheme membership

Why contact costs vary (bivariate) Child characteristics and needs Child continence difficulties (+) Child seen hospital doctor in previous 3 months (+)* Total disability score (+)* Parent outcomes Total quality of life score (+) Total satisfaction score (+)* Scheme characteristics Dedicated funding (+)* Parent rep. Steering Group (-)* Key-worker related Total number of role aspects KW provides (+)* KW makes contact with parent (+)* Total MPOC score (+) KW never positively affects way treated by services (-)* Parent very satisfied with KW (+) Parent wants to see KW more (-)* * Results remaining valid when adjusted for scheme membership

Why contact costs vary (multi-variate) Equation I Childs total disability score (+) Number of role aspects KW provides (+) Parent representation on Steering Group (-)

Why contact costs vary (multi-variate) Equation I Childs total disability score (+) Number of role aspects KW provides (+) Parent representation on Steering Group (-) Equation II Childs total disability score (+) Number of role aspects KW provides (+) Site E (+) Low Adjusted R 2 : c18% cost variation explained. Only explored the cost variation in one of the services used. On average families saw 4 types health and social care services (range 0-9) and 0.8 (range0-3) types of school- or education-based services in the 3 months prior to interview.

Use of other services Services% (n) using Service% (n) using Health care General practitioner Psychologist/psychiatrist Speech therapist Occupational therapist Community nurse Health visitor Hospital doctor Community paediatrician 61% (115) 12% (22) 56% (105) 37% (69) 21% (39) 21% (40) 47% (90) 34% (64) Social care Social worker Voluntary org. worker School/Education Portage/home liaison Behav. support in sch. Learning support in sch. Classroom assistant Other service 24% (45) 11% (20) 9% (17) 5% (10) 27% (51) 24% (46 13% (24) Data from 177 families for use of health and social care, and from 135 families for school/education services

Path Analysis of Satisfaction with Key Worker Service - using costs? Contact cost

Parent interviews Understanding v confusion about key worker role Whole family approach Key worker being accessible and having time Proactive role KW spending time with child KW support in multi-agency meetings The last four have implications not just for the way KW work, but also for costs/budgets.

Effective key worker services Key workers carrying out different aspects of the role A clear job description for key workers Appropriate levels of contact with families Regular training, supervision and support for key workers in their key worker role Some dedicated funding for the service A service manager Time for key workers to carry out the role There is a direct link between many of these factors and cost but the link between improving parent outcomes and costs is not straightforward but through the quality of the support provided.

The key worker role Providing information and advice about services Emotional support Identifying and addressing needs of all family members Coordinating care Improving access to services Speaking on behalf of the family when dealing with services Support in a crisis Providing information about childs condition

Benefits of mixed methods approach Triangulation views of different parties support from different types of data Different methods inform each other – e.g. survey informed questions asked of staff; qualitative data informed quant analysis Aid to interpretation Fun! Different types of data convince different groups of people – putting it all together allows us to persuade more people

Example Quantitative data show importance of job description, qualitative data show why: I suppose my main idea is that its my responsibility to communicate with all other professionals involved...and youd like to feel that youre the first person the family would turn to if theyve got worries. And Id also feel a responsibility in getting the problem sorted, even if it wasnt in my area I would feel that was my job I wouldnt be proactive I think is what Im saying.. in looking for things that there might be difficulties with…Im not checking up on other professionals and their involvement in families, thats not my role, I do not see that as my role. (two key workers in the same service)

Some services consistently provided the families with leaflets and explained the key workers role to them – families understood the role: to ensure that we had every possible service available to us that we were entitled to, to maintain the smooth running of these services once in place and also to provide emotional support if and when required In others, parent understanding varied:well I'm confused about that and what it should be really.. and I don't really know I suppose, I don't really know what that role should be.

Downsides Time Finding researchers able to work with and understand all methods Multiple funders!