1. Evaluating the Outcomes of Social Work Education (OSWE)

2. Aims of the symposium Disseminate the lessons of the OSWE project about the methods and research designs Disseminate the findings from the projects in England and Scotland Discuss a range of effective methodologies Address the challenges of implementing pedagogic research in social work education Consider the future of outcomes based research

3. Symposium programme 10.30 Introduction and overview of OSWE project 11.15 Coffee 11.30 Parallel session A 12.10 Parallel session B 12.50 Lunch 13.45 Parallel session C 14.30 Workshops (x 3) 15.15 Tea 15.25 Plenary

4. Aims of OSWE project To test the feasibility of outcome measures and research designs in evaluating Social Work Education To generate high quality evidence about the effectiveness of methods of SWE To build capacity and capability amongst academics, including service users To use opportunities to compare and contrast practice between programmes

5. What do we mean by outcomes? Change over time Before – after designs (preferably with comparison group) Something measurable

6. Collaborative capacity & capability building model Peer learning through Action Learning Sets Support, advice and mentoring, F2F, e- mail and and by phone (e.g. reviewing draft tools) Site consultations Proformas Blackboard site www.ole.bris.ac.uk Active engagement of a range of stakeholders

7. Outcome 1 (affective or attitudinal) I am interested in methods of evaluating the outcomes of SWE: 1.Strongly disagree 2.Disagree 3.Don’t know 4.Agree 5.Strongly agree

8. Outcome 2 (declarative knowledge) What is a randomised controlled trial? a.Participants are allocated to ‘control’ and ‘comparison’ groups and T1 and T2 scores are compared. b.Allocated randomly to one or more ‘experimental’ groups and a control group; T1 vs. T2 scores compared. c.Frustrated researcher tells uncontrolled, randomised participants to get a grip.

9. Outcome 3 (self-efficacy) I am confident that I can design a methodologically sound evaluation of the outcomes of a SWE course. 1.Strongly disagree 2.Disagree 3.Don’t know 4.Agree 5.Strongly agree

10. Outcome 4 (motivational/behavioural) I am determined to evaluate my course or module next year. 1.Strongly disagree 2.Disagree 3.Don’t know 4.Agree 5.Strongly agree

11. Outcome 5 (procedural/knowledge organisation) I can analyse the challenges involved in evaluating the outcomes of social work education, understanding the relationships between the various elements …and I can demonstrate how to do it.

12. General lessons from the OSWE project (1: planning the work) 1.Support and time is needed to formulate an appropriate, specific research question 2.Selecting and/or adapting appropriate methods for measuring change can be challenging 3.Identifying resources for the research essential 4.Getting ethical approval from the HEI

13. General lessons (2: engaging others) 5.Getting support from colleagues/Head of Dept 6.Establishing the involvement of service users/carers 7.Student engagement needs to be negotiated and carefully planned (opt- out/in, timing, status, promoting learning)

14. General lessons (3) 8.The roles of teacher and researcher may be complementary or may conflict 9.Need to plan access to expertise/learning to analyse and interpret data 10.Finding comparison sites is challenging 11.Giving feedback to others in HEI about findings (and the implications for the curriculum) and about and methodologies.

15. Overall Progress: On measures –Video –Self-efficacy –Vignettes –Concept mapping In working with users and carers Some way to go: In employing rigorous research designs Building the evidence base for SWE

16. Thank you To our sponsors: SCIE, SWAP and IRISS Contacts: j.s.w.carpenter@bristol.ac.ukj.s.w.carpenter@bristol.ac.uk h.c.burgess@bristol.ac.uk Resources: Carpenter J. (2005) Evaluating the Outcomes of Social Work Education, SCIE/SIESWE http://www.iriss.ac.uk/node/88 Burgess, H. and Carpenter, J. (2008) Building capacity and capability for evaluating the outcomes of social work education: creating a culture change. Social Work Education, 27, 8, pp 898 – 912.