1. Evaluation of service user-led role play feedback for social work students Dr Eleni Skoura-Kirk, Lecturer in Social Work Sarah Brown, Senior Lecturer in Social Work Dr Rasa Mikelyte, Research Assistant - CHSS

2. Introduction
Evaluation of service-user led role play interviews as part of the Readiness for Direct Practice module (BA year 1 students)
Examine the development of professional skills (procedural and meta-competencies; Bogo, 2006) for social work students through assessment feedback from service users, practice educators and student self-assessments.
Do role-plays with service users lead to better communication, interpersonal and reflective skills?

3. Skills development through role-plays
Skill development through experiential learning (Kolb, 1984); importance of reflective observation (Cheung and Dalavega 2014; Schon, 1987)
Service user and carer involvement:
involvement in role-plays creates a more ‘authentic’ experience
self-reported improvements in developing empathy, active listening skills, self- awareness and challenge to preconceptions (Skilton, 2011; Duffy, Das and Davidson, 2013; Hitchin, 2016)

4. Role play interviews: design & implementation
OSCE model adapted for social work (Bogo et al, 2006): observed standardized role-play interviews, assessed for development of professional skills
Procedural skills (ability to undertake set tasks to a required level; in this case relationship- forming and professional communication skills)
Meta-competencies (cognitive, critical and self-reflective capacities of the learner)
Formative assessment based on: PI feedback, Student self reflection and PE group feedback

5. Role play interviews: design & implementation
Role-play based on scenario written by Partnership Initiative members (PIs), given to students in advance.
Reflecting PCF Readiness for Direct Practice levels, assessing building rapport, engaging with the individual and identifying some initial assessment information.
Learning and Teaching Enhancement Fund (University of Kent)
Roleplay 1
Roleplay 2
15 minute roleplay between 1 Student and 1 PI member
Feedback collected from Students and PIs soon after
PE-facilitated reflective discussion 1 week later, PE feedback collected
15 minute roleplay between 1 Student and 1 PI member
Feedback collected from Students and PIs soon after
PE-facilitated reflective discussion 1 week later, PE feedback collected
November 2017
March 2018

6. Methodology Students: Participants: 70% 20-30yos 17% 30-40yos
97% Female

7. Methodology Qualitative section: Quantitative section:
Thematic analysis of:
PI qualitative feedback on assessment forms
Student self- assessment forms completed soon after the role plays (qualitative feedback)
PE reflective discussion feedback form (qualitative)
Quantitative section:
PE Survey on roleplay performance
3 items; 10-point semantic differential scale
Student Self-Rating Survey on roleplay performance
5 items; 10-point semantic differential scale
PI Survey on roleplay performance
15 items; 5-point Likert scale
Repeat measures

8. Qualitative Findings

9. 1. Overall improvement Students PIs PEs
“Since the first role play I feel that I have gained many new communication skills, I am a lot more confident when speaking to service users” (S18)
PIs
From: “Student was so nervous that it is very difficult to answer these two questions. She shows signs of future promise but will have to work on her approach to service users and lack of confidence” To: “Student goes from strength to strength. She is confident and capable and knows how to ask the right questions” (P03 for S22)
PEs
“[The students] were more confident regarding asking open-ended questions compared to when they completed the initial role plays” (E02)

10. 2. Diverging emphases of what improved
Students
Procedural &
Explicit
Improvement
PEs
Cognitive &
Theory-to-Practice
Improvement
PIs
Relational & Embodied
Improvement

11. 2. Diverging emphases of what improved
Students: Procedural & Explicit Improvement
PEs: Cognitive & Theory-to-Practice Improvement
PIs: Relational & Embodied Improvement
“I ensured that I was: summarising, reflecting, using minimal encouragers, showing empathy, used empowering speech and my body language was open” (S24)
“More discussion about theories used – strengths-based, Active listening and paraphrasing, Narrative theory, Ecological Theory, Systems Theory and even Solution-focussed” (E01)
“Student was very engaged and completely interested in and focused on what I was saying” (P03)

12. 3. Competing ideas on what improvement is desired
Problem-Solving
Students
“I probably attempted to problem solve too early, rather than establishing a relationship/rappo rt with the service user” (S27)
PEs
“We discussed the temptation to problem- solve and offer solutions before the assessment stage has been completed. It was explored that there could be a risk of not only raising unrealistic expectations, but also not listening to the service user” (E02)
PIs
“Willingness to find out what can be done.” (P07)
“Very quick to think of solutions” (P05)

13. Quantitative Findings

16. Results Time 1 Time 2 M SD df t-test PE Rating Cognitive Development
Time 1
Time 2 M SD df
t-test
PE Rating
Cognitive Development
6.63
1.19
7.97
.67 29
-7.62***
Reflective Development
7.80
.41
-1.43
Affective Development
8.10
.86
6.87
1.14
-4.85***
Student
Self-Rating
Performance
6.70
1.44
7.48
1.55 26
-2.01
Knowledge of theory
6.29
1.78
7.93
1.33 27
-4.65***
Confidence
6.46
1.80
1.54
-4.27***
Helpfulness of Reflection
8.89
1.83
9.18
1.52
-.90
Need to develop skills
6.15
2.07
4.44
2.08
3.23**
PI Ratings
Introduction
4.07
1.16
4.76
.44 28
-3.10**
Explanation for meeting
1.08
4.40
.72
-1.44
Explanation of agenda
3.83
1.12
4.37
.77
-2.44*
Sticking to time
4.20
.93
4.60
.68
-2.56*
Listening to PI
4.80
-3.03*
Allowing for PI views
4.50
.63
4.77
.50
-2.28*
Understanding PI views
4.14
.83
4.55
.57
-2.35*
Look out for PI comfort
3.89
.85
4.48
.58
-3.65**
Treating PI with respect
4.66
.61
4.83
.38
-1.54
Care for PI feelings
4.30
.84
4.73
.52
-2.90**
Asking relevant questions
4.17
.87
-1.84
Avoiding jargon
4.43
-2.19*
Preparedness for meeting
4.24
.51
-1.88
Ending the meeting
.75
-1.86
3.90
.96
.56

17. Results Time 1 Time 2 M SD df t-test PE Rating Cognitive Development
Time 1
Time 2 M SD df
t-test
PE Rating
Cognitive Development
6.63
1.19
7.97
.67 29
-7.62***
Reflective Development
7.80
.41
-1.43
Affective Development
8.10
.86
6.87
1.14
-4.85***
Student
Self-Rating
Performance
6.70
1.44
7.48
1.55 26
-2.01
Knowledge of theory
6.29
1.78
7.93
1.33 27
-4.65***
Confidence
6.46
1.80
1.54
-4.27***
Helpfulness of Reflection
8.89
1.83
9.18
1.52
-.90
Need to develop skills
6.15
2.07
4.44
2.08
3.23**
PI Ratings
Introduction
4.07
1.16
4.76
.44 28
-3.10**
Explanation for meeting
1.08
4.40
.72
-1.44
Explanation of agenda
3.83
1.12
4.37
.77
-2.44*
Sticking to time
4.20
.93
4.60
.68
-2.56*
Listening to PI
4.80
-3.03*
Allowing for PI views
4.50
.63
4.77
.50
-2.28*
Understanding PI views
4.14
.83
4.55
.57
-2.35*
Look out for PI comfort
3.89
.85
4.48
.58
-3.65**
Treating PI with respect
4.66
.61
4.83
.38
-1.54
Care for PI feelings
4.30
.84
4.73
.52
-2.90**
Asking relevant questions
4.17
.87
-1.84
Avoiding jargon
4.43
-2.19*
Preparedness for meeting
4.24
.51
-1.88
Ending the meeting
.75
-1.86
3.90
.96
.56

18. Results
Change between Time 1 and Time 2 N M SD
Correlations 1 2 3
1. Change in PE Ratings 30
1.00
1.06
2. Change in Student Rating 27
.50
1.22
2.55
3. Change in PI Rating 26
.42
.63
-.12
-.02
Most ratings showed improvement at Roleplay 2, whether they were made by PIs, PEs or the students (based on t-test results)
However, PI, PE and Student self-ratings did not go together
Student self-perceived improvement or deterioration in roleplay performance was not linked to that seen by the PIs or the PEs

19. So…
Do role-plays with service users lead to better communication, interpersonal and reflective skills?

20. Key messages
Evidence suggests overall improvement in students’ professional skills.
The repetition of the task contributed to increased confidence, allowed exercise and refinement of procedural skills and linking theory to practice.
Three-fold assessment promotes a holistic approach to the task (procedural/cognitive/relational).
Competing ideas: importance of service user and carer involvement in social work education, avoiding exclusive reliance on professional narratives.

21. References
Bogo, M., Regehr, C., Woodford, M., Hughes, J., Power, R. & Regehr, G. (2006) ‘Beyond competencies: field instructors’ descriptions of student performance’, Journal of Social Work Education, vol. 42, no. 3, pp. 579–593.
Cheung, M. and Dalavega, E. (2014) Five-way Experiential Learning Model for Social Work Education. Social Work Education, vol 33, no. 8, pp
Duffy, J., Das, C. and Davidson, D. (2013) Service User and Carer Involvement in Role-plays to Assess Readiness for Practice, Social Work Education, 32(1), pp
Hitchin, S. (2016) Role-played interviews with service users in preparation for social work practice: exploring students’ and service users’ experience of co- produced workshops, Social Work Education, 35:8,
Kolb, D. A. (1984) Experiential Learning. Englewood Cliffs: Prentice Hall.
Schon, D. (1987) Educating the Reflective Practitioner, Jossey Bass, San Francisco.
Skilton, K.J. (2011) Involving Experts by Experience in Assessing Students’ Readiness to Practise: The Value of Experiential Learning in Student Reflection and Preparation for Practice, Social Work Education, 30(3), pp

22. Thanks!
Any questions?
You can contact us at: