1. Improving Selection to the Foundation Programme
Parallel Recruitment Exercise

2. Selection to the Foundation Programme – improving and evolving
FP 2005 Foundation Programme introduced: local paper-based applications FP 2006 National timetable and standard application introduced: 50% paper-based applications + 50% online applications FP 2007 – 100% of applications online using ‘white present space’ application form questions and academic quartiles

3. Evolution
2009 DH commissioned a review of selection methods. The Improving Selection to the Foundation Programme project was set up and overseen by the Medical Schools Council 2010/2011 New selection methods were piloted successfully FP 2012 Full-scale Parallel Recruitment Exercise (PRE) FP 2013 New selection methods implemented
The Department of Health commissioned a review of the current selection methods, not because they didn’t work, but because there were concerns about the long term availability of suitably different ‘white space’ questions, and to ensure that the most current research evidence and selection methods are used. The review included academic literature reviews of high-stakes selection, extensive stakeholder consultation, advice from an international panel of experts and an independent cost benefit analysis.
The new selection methods were piloted with 1100 students in , in 15 UK and two non-UK schools. The pilots have demonstrated the evidence that the new selection methods are the most fair and effective method of selection to the Foundation Programme.
Applicants to start the Foundation Programme in August next year will apply using ‘white space’ questions and quartiles; the new methods will be used for applications to FP2013 onwards

4. What are the new selection methods?
Situational Judgement Test (SJT)
SJTs will replace the ‘white space’ application form questions
This is an invigilated, machine markable test in exam conditions
The SJT will consist of around 70 questions in 2 hours 20 mins
Educational Performance Measure (EPM)
The EPM will replace the academic quartile scores. The EPM score is comprised of three elements:
medical school performance in deciles [SEE accompanying ppt on basket of assessments, quartiles and deciles]
additional degrees
academic achievements
The EPM and SJT will each be worth 50 points from a 100 point application score
The EPM reflects academic performance at medical school, and additional academic achievements – evolving from academic quartiles and Q1 of the ‘white space’. There are no points for extra-curricular activities – it would be impossible to compare care for a family member, with committee leadership, with volunteering - however the personal gains from taking part in extra-curricular should be apparent through answers to the SJT.
The SJT assesses the attributes of the national person specification – as the current white space questions do
The evidence shows that the SJT and EPM in combination assess the clinical and non-clinical knowledge, skills, and personal attributes to be effective in the role of an FY1 doctor
The change is about evolution, not revolution. The EPM and SJT are changes to the way that the application score is produced; but applicants will continue to declare their preferences for foundation schools, make linked applications, special circumstances and so on.

5. How will recruitment work in the future?
FP 2012
FPAS runs as normal + new recruitment process trialled
through the Parallel Recruitment Exercise (PRE)
SJT: All students are expected to participate in a shortened version of the
SJT– the exam will only be one hour and will consist of 30 questions
EPM: Medical school staff will calculate EPM deciles for each student
IMPORTANT: Scores from the PRE EPM and SJT will not impact your FPAS
score - and you will receive feedback
FP 2013
All applicants to the Foundation Programme, including to Academic
Programmes, will take the SJT, and will receive an EPM score
The Parallel Recruitment Exercise (PRE) will not affect the FPAS score for final year students applying this year to the Foundation Programme 2012.
There will continue to be a separate selection and appointment process for Academic Foundation Programmes; however ALL applicants will be required to complete the national application process, including the SJT and EPM.

6. What is the PRE?
New selection methods will be trialled alongside the normal selection methods during this year’s national FP recruitment round
All final year medical students are expected to participate in the SJT, in addition to completing their FPAS forms
The PRE is the final step in ensuring the selection methods can be consistently and robustly applied before implementation for FP2013
Parallel Recruitment Exercise – running in all UK medical schools during the 2011/2012 academic year. Final year medical students only – expected to take part in a shortened one-hour SJT
The PRE is an opportunity to trial the SJT and EPM on a national scale, and to provide additional evidence that the SJT and EPM are the way forward (n.b. the pilots have demonstrated that the SJT is a fair and effective measure of the aptitude for the FY1 role, with 1100 participants – the PRE is on a national scale) – students will be asked for their views and feedback

7. What is a Situational Judgement Test?
SJTs are:
a test of aptitude
designed to assess the professional attributes expected of a Foundation doctor
based on a detailed job analysis of an FY1 doctor
SJT questions assess your judgement by presenting you with
challenging situations you are likely encounter at work during the first
year of an integrated Foundation Programme
SJTs are an invigilated assessment of professional judgement and the attributes expected of the Foundation doctor.
The SJT is not something students can revise for, but these are skills and attributes that they should exhibit, and that they will have been made aware of throughout their time at medical school, on placements and through e.g. Tomorrow’s Doctors.
SJTs are used in selection to GP training, trialled for selection to five other specialties, and used by FBI, Fire Service, police, civil service

8. Job analysis of FY1 doctor
Commitment to professionalism
Coping with pressure
Effective communication
Learning and professional development
Organisation and planning
Patient focus
Problem solving and decision-making
Self-awareness and insight
Working effectively as part of a team
The findings of a detailed Job Analysis (including shadowing of FY1s, 50+ interviews with FY1s and colleagues working with FY1s, as well as literature reviews) will be published.
The Job Analysis identified more than 100 behaviours which can be grouped into the nine ‘domains’ listed on the slide in alphabetical order. The individual SJT items are mapped against these domains, and the test will include questions across the breadth of these domains.

9. Why change from ‘white space’ questions?
What students say:
Some applicants get help writing their answers from parents/friends who are doctors and it is not fair to those who don’t get help
Model answers can be bought on the internet
Very time-consuming and stressful to complete during a busy time
‘White space’ questions are an exercise in creative writing
Perceived variability in scoring
Students have raised concerns during the recruitment feedback surveys about ‘white space’ questions, particularly:
risk of plagiarism – with a 2 week window within which applicants can answer the questions
Anecdotal feedback that ‘white space’ questions are an exercise in creative writing

10. Why change from ‘white space’ questions?
What the UKFPO and deaneries say:
Difficult to ask questions about teamwork, professionalism, etc in sufficiently different ways
The more similar the questions are, the more model answers and ‘answer writing’ workshops will be available resulting in a narrow spread of scores
Difficult to differentiate between candidates with similar scores for selection into the Foundation Programme
Scoring is very resource intensive
Whilst the current system currently works well, there are a number of concerns that need to be addressed including:
Longevity of white space questions – there’s a limit to how many variations of questions can be asked
Costly in terms of consultant time to score white space questions – raises the question ‘is it cost effective?’

11. Why change to SJTs? Exam conditions – therefore fair for everyone
Research evidence to support use in this context
Sufficient questions can be developed for long-term use
Access to example questions and answers prior to taking SJT
Marks for near misses
Able to differentiate between applicants
Within medical selection, SJTs have been shown to be a reliable and valid method of selection
Evidence shows ability to predict performance in the role
SJTs are increasingly popular in large-scale selection (in medical selection in the UK and in other disciplines – police, FBI, fire service)
Students will take the SJT in exam conditions and so everyone will have an equal chance to do well.
There is research evidence to support the use of SJTs and the range of different scenarios an FY1 might encounter means that a sufficient number of questions can be developed in order to use SJTs in the long term.
Students and schools will have access to example questions and answers to prepare them with the format of questions, and how the answers are scored
The scoring key allows for near misses for the ranking questions – so for example if ABCDE is right answer, and student answers BACDE, would score nearly full marks
Pilots so far have shown that SJTs are able to differentiate between students – some students felt that would result in everyone getting similar score but this was shown not to be the case.

12. Example SJT Questions There are two question formats:
Rank the five responses in the most appropriate order
Choose the three most appropriate responses from eight
You should answer what you ‘should’ do in the scenario
described, not what you ‘would’ do
SJTs are a test of aptitude and are designed to assess the professional attributes expected of a Foundation doctor. There are two question formats:
 Rank five possible responses in the most appropriate order
Select the three most appropriate responses for the situation (from eight possible options)   
Students must answer what they ‘should’ do in the scenario described, not what they ‘would’ do. This is because research into SJT shows that questions asking an applicant what they ‘would’ favours the more ‘test wise’ – plus the GMC has an emphasis on probity

13. Example Question 1 – rank response
Mr Johnson is admitted with a minor groin abscess requiring surgical drainage although he is otherwise well and has full mental capacity. Four hours prior to surgery Mr Johnson informs a nurse that he wishes to self discharge as he says he is due in court. Mr Johnson’s next of kin are aware that he has been admitted for surgery and that he is due in court. The nurse asks you to speak to him. Rank in order the following actions in response to this situation (1= most appropriate; 5= least appropriate). A Allow Mr Johnson to leave but advise him to see his General Practitioner if there are further problems B Prevent Mr Johnson from leaving by phoning security C Explain to Mr Johnson the risks of leaving without treatment D Telephone Mr Johnson’s next of kin to ask them to try and persuade him not to leave E Allow Mr Johnson to leave but ask him to return to the hospital as soon as possible

14. Answer to Question 1
C Explain to Mr Johnson the risks of leaving without treatment E Allow Mr Johnson to leave but ask him to return to the hospital as soon as possible D Telephone Mr Johnson’s next of kin to ask them to try and persuade him not to leave A Allow Mr Johnson to leave but advise him to see his General Practitioner if there are further problems B Prevent Mr Johnson from leaving by phoning security Communicating to Mr Johnson the risks of leaving is important as he may not be aware of these risks (C). However, it is the patient’s choice as to whether he remains in hospital. Informing him that he should return to the hospital is important (E). Mr Johnsons’ next of kin are aware of the situation and may be able to persuade him to stay (D). By asking Mr Johnson to see his GP, instead of returning to the hospital, you would be passing responsibility for his care (A), however this is more appropriate than forcing a patient to stay against his wishes (B).
Correct answer (CEDAB) would receive a maximum of 20 marks – 4 marks per answer in the right order
If answered ECDAB (only first two the wrong way round), would score almost full marks
If answered B as the most appropriate (when it is the least) but all others in order i.e. BCEDA, would receive 0 points for answer ‘B’, but 3 out of 4 for each of the others i.e. a total of 12 out of 20 marks.
n.b. the total marks available for an SJT paper will be scaled so that the SJT score will be worth 50 points on the FPAS application form, out of the total 100 points

15. Example Question 2 – choose best 3
You review a patient on the surgical ward who has had an appendicectomy done earlier on in the day. You write a prescription for strong painkillers. The staff nurse challenges your decision and refuses to give the medication to the patient. Choose the THREE most appropriate actions to take in this situation A Instruct the nurse to give the medication to the patient B Discuss with the nurse why she disagrees with the prescription C Ask a senior colleague for advice D Complete a clinical incident form E Cancel the prescription on the nurse’s advice F Arrange to speak to the nurse later to discuss your working relationship G Write in the medical notes that the nurse has declined to give the medication H Review the case again

16. Answer to Question 2
B Discuss with the nurse why she disagrees with the prescription C Ask a senior colleague for advice H Review the case again Ensuring patient safety is key to this scenario. It is important to discuss the nurse’s decision with her as there may be something that you have missed when first reviewing the patient (B). Therefore it would also be important to review the patient again (H). Also relating to this is the importance of respecting the views of colleagues and maintaining working relationships, even if there is disagreement. As there has been a disagreement regarding patient care, it is important to seek advice from a senior colleague (C).
In this example, students will receive 4 marks for each correct answer. However, if they put more than 3 options are selected they will get 0 marks as they have not answered the question. i.e. a score of 0, 4, 8 or 12 for this question.
n.b. the total marks available for an SJT paper will be scaled so that the SJT score will be worth 50 points on the FPAS application form, out of the total 100 points

17. Taking the PRE SJT at your school
All final year medical students are expected to complete their FPAS form PLUS a one hour invigilated SJT
SJTs will:
take place at Nottingham Medical School on Friday 9 Dec. Your school will confirm the date, time and venue
give you the opportunity to practice taking an SJT, which you are likely to encounter for specialty selection
provide you with your score in relation to all other students who have taken the test (feedback in April 2012)
NOT have any bearing on your FPAS score, nor on your allocation to foundation school or programme
All students are expected to take part in a one hour SJT in addition to completing their FPAS application
Applicants will be given feedback on their score in April 2012 via the FPAS system, although this will have no effect on their application score or foundation school allocation
There should be provision for students with additional needs e.g. extra time

18. Benefits of taking part in the PRE SJT
It is good preparation for specialty selection
You will get feedback on how you scored
You are helping to form national recruitment policy
You are expected to take part unless you have permission to be exempt
You may win an i-pad!
PRACTICE! Taking part in the PRE SJT provides students with an excellent chance to familiarise themselves and practice this style of question format before encountering it for real during specialty selection
They will receive feedback on their score
They will be helping to form the national recruitment policy – and their views and input are extremely important to ensure a fair and evidenced based system is implemented
All final year students are expected to take part – permission may be granted if they are away on elective – have to explain to a senior medical school staff member know in writing
There are 5 prizes of an i-pad (drawn at random from all participants)

19. SJT for selection to FP2013 onwards
All applicants to the Foundation Programme will be required to complete an SJT in the UK
The SJT will be run by UK medical schools on two national dates, to allow for electives and extenuating circumstances
The SJT will last 2 hours 20 minutes, and will consist of around 70 questions, including a small number of ‘pilot’ questions which will not count towards the application score
The SJT will be completed on paper, with the possibility of computer delivery in future
There will be provision for access arrangements e.g. extra time
The two national dates will be between November and January – to be announced

20. Why change to EPM?
A clear framework with agreed principles will be used to calculate the EPM, ensuring that it is fair, transparent and consistent across the schools of the UK
Splitting cohorts into deciles rather than quartiles provides a wider spread of scores, which makes it easier to differentiate between applicants, and will be more fair for applicants at the margins
It makes more sense for all the academic components of the application to be one part of the application, rather than being split between ‘white space’ questions (Q1) and academic quartile ranking
It is not possible to compare performance of students in different medical schools, as schools offer different assessments at different times throughout the course – and the FPAS application is completed before finals. The most fair way to recognise achievements at medical school is thus relative to applicants from the same medical school.
The way that quartiles are calculated does vary between schools; in future all schools will be required to follow an agreed framework, and to publish the way that they calculate points for performance at medical school. The EPM will improve consistency and transparency.
Deciles instead of quartiles – more fair
Brings together points for additional academic achievements, which is currently Q1 of the ‘white space’ questions
Recommendations on the best way forward, using evidence gathered from pilots during the 2010/2011 academic year.

21. How is the EPM calculated?
Score produced by applicant’s medical school to reflect achievement
and performance compared to rest of cohort
EPM = 3 parts (maximum 50 points):
Medical school performance in deciles (34 – 43 points)
E.g. Top 10% = 43; Top 20% = 42; etc
Additional degrees (max 5 points)
Educational achievements (presentations, prizes and publications (max 2 points)
Schools will consult with students about how the decile points for the EPM will be calculated. This will then be published online
The EPM reflects a student’s performance at medical school on summative assessments, with all schools (UK and non UK) using the same framework which outlines the rules that each school must follow when calculating the EPM decile score. Schools will publish the way that they calculate deciles according to this framework, specifying the assessments and weightings used locally.
Deciles will be worth points from the 50 points for the EPM
As with the current application form, applicants are each eligible for additional points to recognise degrees, presentations, prizes and publications – up to 7 points in total, from the 50 points for the EPM
There are no points for extra-curricular activities – it would be impossible to compare caring for a family member, with committee leadership, with volunteering - however the personal gains from taking part in extra-curricular should be apparent through answers to the SJT.

22. 1. Medical school performance (34-43 points)
Your medical school will use your results from a number of
assessments up to the end of the penultimate year at medical school
to calculate your decile score, as outlined in the EPM Framework
published on
The EPM Framework sets out the principles for calculating deciles,
which are listed below. Assessments must:
be summative
(in combination) cover clinical knowledge, skills and performance
(in combination) cover non-clinical performance
cover all aspects of the curriculum assessed up to the end of the penultimate year at medical school
represent the average performance of applicants over time
include written and practical assessments
Students will be assessed and ranked on their medical school performance, according to the EPM framework.
Schools must consult with students, and will publish the agreed way that they calculate deciles according to this framework, specifying the assessments and weightings used locally.

23. 2. Additional degrees (max 5 points)
A maximum of 5 points for degrees awarded by time of application to Foundation Programme:
Previous degree
Number of points
Doctoral degree (PhD, DPhil, etc) 5
Masters degree
1st class honours degree
Bachelor of Dental Science (BDS)
B Vet Med 4
2.1 class honours degree
1st class intercalated degree which does not extend the degree programme 3
2.2 class honours degree
2.1 class intercalated degree which does not extend the degree programme 2
3rd class honours degree
Unclassified or ordinary degree
2.2 class intercalated degree which does not extend the degree programme 1
Primary medical qualification only
3rd class intercalated degree which does not extend the degree programme
If an applicant holds more than one degree at the time of application to the Foundation Programme, they should provide evidence of the degree that will achieve the highest number of points.

24. 3. Educational achievements (max 2 points)
A maximum of 2 points will be awarded for national prizes, presentations and publications:
Educational achievements
Number of points
Prizes
1st prize – National /international educational prize 1
Presentations
Oral presentation at a national or international conference
1st named author in a poster or presentation at a national or international conference
Publications
Educational research paper published in a peer-reviewed journal
Maximum number of points available 2
If an applicant has more than one publication, prize or presentation, they will receive a maximum of one point for any of the three categories individually; a maximum of two points in total.

25. Parallel Recruitment Exercise - EPM
Schools will consult with students about how the decile scores will be calculated SEE accompanying PPT
Your medical school administrator will calculate your decile score which
will be analysed alongside your Question 1 scores (additional degrees
and educational achievements). You will be told your score in April 2012.
Your decile score will:
be calculated by your medical school using the methods they are planning to use for FP2013
be used confidentially for analysis
NOT have any bearing on your FPAS application score, nor will the information be used to determine your allocation to foundation school or to programme
In the event of failed finals/re-applying to the FP2013, the EPM decile score calculated this year will count
Schools will be consulting with students during the autumn term 2011, to agree the way that decile points will be calculated.
In the same way that it is up to medical school staff to calculate quartiles, it is the schools who will calculate deciles – students don’t have to do anything extra.
The EPM score will not affect applications to the Foundation Programme in any way – however students will be told their decile points for information, in April 2012 (when they will also receive feedback on their SJT performance)
If a student fails the year or re-applies to the FP2013, their EPM decile score calculated this year will be used (as their performance up to the penultimate year will not change) – but points for additional degrees, prizes etc will be at the point of application.

26. Selection to FP2012 FPAS application to be completed as normal
Participation in the Parallel Recruitment Exercise will not affect your FPAS application in any way
Situational Judgement Test (SJT)
You are expected to take part in a shortened 1 hour SJT
You will receive feedback on your SJT score in April 2012
Educational Performance Measure (EPM)
Your school will consult on how deciles will be calculated in future
You will receive feedback on your decile score in April 2012
SUMMARY

27. Selection to FP2013 onwards
Situational Judgement Test (SJT) – max 50 points
SJTs will replace the ‘white space’ application form questions as a measure of meeting the national person specification
SJTs assess aptitude for the FY1 role
A machine markable, multiple choice test taken in exam conditions
The SJT will consist of around 70 questions in 2 hours 20 mins
Educational Performance Measure (EPM) – max 50 points
The EPM will replace academic quartiles as a measure of clinical and non-clinical skills, knowledge and performance
The EPM score is comprised of three elements:
medical school performance in deciles (34-43 points)
additional degrees (up to 5 points)
academic achievements (up to 2 points)
SUMMARY

28. More information ISFP website – www.isfp.org.uk Discussion forum
E-bulletin updates
Project handbook
FAQs
Facebook
YouTube (background info)
UKFPO student representative at your school
s from medical schools – keep a look out in your inbox!
Student views are hugely important. These are some of the different ways students can keep update with the project, find out more and send us their comments.

29. Nottingham contact: john.whittle@nottingham.ac.uk
Any questions?
If you cannot answer any questions, please contact us with these and we will respond directly to you.
Nottingham contact: