Assessment and Learning in Practice Settings (ALPS) © Student’s perception of competence to practice Dr J D Cortis Assessment and Learning in Practice Settings Centre for Excellence in Teaching and Learning

Assessment and Learning in Practice Settings (ALPS) © The work of ALPS One of 74 CETLs funded by HEFCE, spans 16 health and social care professions and 5 universities Aims to ensure that: “Students graduating from courses in health and social care are fully equipped to perform confidently and competently at the start of their professional careers”

Assessment and Learning in Practice Settings (ALPS) © The BORG (no, not that one!) Baseline and Outcomes Research Group Wanted a “before” and “after” ALPS view of competence and confidence – with usual caveats Reviewed current work on measures of competence and competence Did not find a measure that would work across all 16 ALPS professions – and did not find many actual ‘measures’ at all Developed Competency in Practice Assessment (CIPA) tool

Assessment and Learning in Practice Settings (ALPS) © Measuring Competence? Context (1) In spite of overwhelmingly positive outcomes and reports of recent QAA and NHS Major Reviews and generally positive reports from ongoing professional and regulatory body programme approval, monitoring and review processes, there remain sporadic outbursts of anecdotal evidence of employer dissatisfaction with the competency of new graduate health and social care professionals.

Assessment and Learning in Practice Settings (ALPS) © Measuring Competence? Context (2) The ability to demonstrate that new graduate professionals are competent and confident at the point of qualification and initial professional registration therefore remains a significant challenge

Assessment and Learning in Practice Settings (ALPS) © The CIPA Tool Development Usable by 16 different health and social care professions (in the first instance) – hopefully more as time progressed – ‘generic’ but meaningful language Once written, consulted with an academic representative from each of the 16 ALPS professions – some negotiation over language, then final version agreed Online to ensure widest coverage

Assessment and Learning in Practice Settings (ALPS) © Pilot Phase 1 Involved three pre-registration cohorts at Leeds Metropolitan University who were mid-way through their final year of study. 33 students completed the pilot This was done in order to calibrate the data collection tool (using Rasch analysis) as we needed to be able to identify students who did not feel ‘confident’ or ‘competent’ to practice as well as those who did. In theory, students who were not qualified would not score highly across the tool.

Assessment and Learning in Practice Settings (ALPS) © Approach Perceptions of Confidence and Competence Rather than a ‘snapshot’ of perceptions of confidence and competence, a longitudinal view was preferred Led to decision to use CIPA Tool: - At point of graduation - 6 months post graduation - 12 months post graduation

Assessment and Learning in Practice Settings (ALPS) © Pilot Phase 2 The second phase of the pilot involved attempting to implement the tool with all graduating cohorts across the institutions and courses involved. Whilst we were obviously keen to collect data in this pilot, we also wanted to explore what issues would arise when attempting to implement such a large scale, cross-institutional study. 258 students from 4 universities completed the CIPA This data was also used to feed in to the first Rasch analysis to look at the reliability of the data collection tool

Phase 1

Phase 2

Phase 3

Assessment and Learning in Practice Settings (ALPS) © Deal with cultural issues

Assessment and Learning in Practice Settings (ALPS) © Be open to feedback from others

Assessment and Learning in Practice Settings (ALPS) © Establish a good relationship

Assessment and Learning in Practice Settings (ALPS) © Student comments; “ Overall I thought the course was somewhat difficult at times, however I found that I have learnt more than I thought I actually did as now I am in the position where I am teaching other students” “I was thoroughly prepared for both the hard work and every single aspect of the (named course) profession…I now feel completely confident and competent in my abilities” “When I started my course one of the lecturers said that we would be different people after 3 years. She is absolutely correct, I am a different person. I am now doing my dream job as a nurse and I’m earning a great wage also”

Assessment and Learning in Practice Settings (ALPS) © Summary The CIPA tool has highlighted “big” issues for the ALPS partners e.g. importance of practice It has also highlighted the “small” issues which can inform future curriculum development e.g. dealing with cultural issues It is an effective tool, allowing us to pool powerful data We can see trends over time – more work required to collect data following qualification.