1. Improving functional skills qualifications Bryan Horne Friday 13th February 2015

2. Improving functional skills qualifications 1.Functional skills – background 2.Why we did our work 3.What we did 4.Improvements being made 5.What next

3. BACKGROUND The market, assessment and use

4. Functional skills - history Functional skills qualifications are applied qualifications in English, mathematics and Information and Communications Technology (ICT). Available from entry level through to level 2 in England. Allow students to demonstrate they have practical skills in literacy, numeracy and IT that help them to live and work confidently, effectively and independently. Developed and piloted from 2007-2010 by awarding organisations and the Qualifications and Curriculum Authority. The content and the regulatory criteria were published in 2009 for qualifications that were available for first teaching in 2010.

5. Functional skills – criteria The ‘skills standards’ are set out in the regulatory criteria - based on the Adult Literacy and Numeracy Curriculum. The Criteria were reissued in 2011 and are available on Ofqual’s website: https://www.gov.uk/government/collections/functional- skills-qualifications-requirementshttps://www.gov.uk/government/collections/functional- skills-qualifications-requirements Made up of components: ICT and mathematics have a single component and English three components: reading; writing; and speaking, listening and communication. ‘Size’ of qualification - 45 guided learning hours. Assessment is on a ‘pass-fail’ basis – there are no grades. The criteria set out the skills that must be demonstrated to pass each subject. Assessment methodologies used by different awarding organisations vary.

6. Functional skills - market Used for apprenticeships, in adult education, and as a free- standing qualifications for 14-19 year olds. Some awarding organisations also cater for their use at Key Stage 3 (11-14 year olds) as a ‘stepping stone’ to a GCSE, although the qualifications were not designed for this purpose. Represent ~7% of all regulated qualifications taken in England; more functional skills qualifications are awarded than any other type of qualification apart from GCSEs. Number of qualifications achieved has increased from just under 300,000 certifications per year in 2010/11 to just over a million in 2013/14.

7. Functional skills - market Functional skills certifications 2009-14 at entry level, level 1 and level 2 Source: Ofqual certification data 2009-2014

8. Functional skills - market Functional skills qualifications: market share 2013-14 Source: Ofqual certification data 2009-2014

9. WHY WE DID OUR WORK Our rationale

10. Functional skills – high profile

11. Functional skills – rationale for our work Validity of qualifications is at the heart of the way we regulate. A valid qualification effectively and reliably measures what it is designed to measure. With a clear purpose established in consultation with end-users – employers, or higher or further education providers. Purpose drives content and the assessment methodology used Content, assessment and performance standards must be consistent, so that the level of skill needed to gain a particular grade or pass is comparable over time and between awarding organisations. Standards can come under pressure when qualifications are put to a use for which they are not designed. Principles as relevant to functional skills as to any other qualifications – a key driver for our work.

12. WHAT WE DID How we approached our work

13. Functional skills – our approach From May to August 2014 we worked with each of the 13 awarding organisations offering all 28 level 2 English and mathematics functional skills qualifications. We looked at assessment materials from each of the qualifications. We met each of the awarding organisations during and following our review to discuss their systems, resources and processes. Three of the 16 awarding organisations offering functional skills do not offer English or mathematics qualifications at level 2.

14. Functional skills – our approach * In 2011 EDI merged with Pearson. Both awarding organisations offered functional skills qualifications. Pearson EDIs functional skills qualifications will be discontinued by mid-2015. Awarding organisation City & Guilds of London Institute Pearson Education Ltd (including Pearson EDI)* AQA Education Ascentis EAL (Excellence, Achievement and Learning Limited) IQ NCFE NOCN (National Open College Network) OCR SQA (Scottish Qualifications Authority) Skillsfirst Awards Ltd VTCT WJEC

15. Functional skills – our approach Source: Ofqual certification data 2009- 2014

16. Functional skills – our approach A four-week call for evidence on functional skills qualifications during June 2014. A short survey was publicly available on our website and advertised to a range of stakeholders. Questions were designed to gather stakeholder views about how far the qualifications met their intended purpose of improving skills - for life and learning, and also how far they met employers' needs.

17. IMPROVEMENTS BEING MADE

18. Functional skills – improvements Key areas where improvements are being made Examples of how awarding organisations are doing this Changes students, teachers and others might see 1. Improving the quality of assessment materials, including:  More higher level / stretch questions  Better coverage of the range of the skills as specified in the regulatory criteria  Better aligned question papers and mark schemes Reviewing and strengthening test development procedures Development of improved test specifications Strengthened quality assurance processes for assessment materials Increased pre-testing of questions Questions covering a wider range of areas and in some areas providing additional challenge More realistic context to questions More open questions that allow students to demonstrate problem- solving skills Fewer question paper errors

19. Functional skills – improvements Key areas where improvements are being made Examples of how awarding organisations are doing this Changes students, teachers and others might see 2. Reducing the risks of malpractice and maladministration  Increased controls over internal (school and college- based) assessment of speaking, listening and communication (SLC)  Increased security of assessment materials Review and update of security policies and procedures Risk-based sampling of live SLC assessment and risk-based monitoring visits Increased monitoring of SLC results, particularly in comparison to reading and writing elements, to enable identification and investigation of test centres with unexpected results Improved standardisation procedures between different centres New training and procedures for moderators More regular refreshes of test content, for example through the introduction of a paper / item bank Reviewing whether the approach to test windows might put the security of test materials at unacceptable risk Updated guidance to schools and colleges on security of assessments Updated guidance on internal assessment of SLC More observation of SLC assessments More centre visits by awarding organisations More regular refreshes of tasks and question papers Possible reduction in the length of test windows

20. Functional skills – improvements Key areas where improvements are being made Examples of how awarding organisations are doing this Changes students, teachers and others might see 3. Improving standard setting procedures Greater scrutiny at awarding to make sure skills standards are effectively met; for example, more full script inspections and borderline checks at panels Revised exemplar materials for centres Additional examiner training Appointment of separate principal examiners for each subject Updated guidance to centres

21. Functional skills – improvements Key areas where improvements are being made Examples of how awarding organisations are doing this Changes students, teachers and others might see 4. Improving evaluation of whether qualifications are meeting user needs Commissioning user research Development and implementation of a stakeholder engagement plan Analysis of destination data and tracking of candidate progression Introduction of subject reviews End-users have greater opportunity to feedback views to awarding organisations

22. Functional skills – call for evidence A significant level of confidence in the qualifications, and that they met employers’ needs. Support was often due to the value attributed to the real-life focus of assessments. The four features of functional skills qualifications most often mentioned as ‘very important’ by respondents were: consistency of marking (91%), prompt delivery of results (90%), accessible tests (88%), and frequent opportunities to take the test (84%). Analysis of responses identified themes for change or improvement that could be made in several areas. Given the numbers that responded and the self-selecting nature of the sample, we cannot assume that responses are representative.

23. WHAT NEXT Next steps and future plans

24. Next steps - 1 Awarding organisations are working to improve each of their qualifications – based on feedback from our work. We will take additional steps to help secure greater consistency of standards between different awarding organisations. We will: –set out guidance on good and poor practice in functional skills question design, both for on-line and paper-based approaches, and –work with awarding organisations to secure their commitment to a consistent approach to setting and maintaining standards between different awarding organisations. We will also consider the impact of flexible and on-demand assessment on standards and comparability.

25. Next steps - 2 In the autumn we will carry out a further review of functional skills qualifications and we will report our findings. We will follow up on the changes made by awarding organisations and check whether they are fully meeting our regulatory requirements. We will not limit our audit to level 2 English and mathematics but will look across subjects and levels. Where there is evidence that our requirements are not being met, we will consider formal regulatory action.

26. Next steps -3 Should we find that the provision of additional informal guidance is not sufficient to achieve greater validity and consistency of standards, we will consider whether we need to take further action. We will review the evidence from our audit, including input from stakeholders, and the findings from the ETF review, to determine how we can best secure the quality and validity of functional skills qualifications in the longer term.

27. Questions Bryan Horne Associate Director Standards for Vocational Qualifications and Apprenticeships, Ofqual T: 02476 716729 M: 0791 2775036