1. Update on Sector Reviews
Cassy Taylor
Associate Director (Vocational Qualifications)

2. Aim of sector reviews
To identify whether qualifications, and the qualifications system are meeting the needs of employers and learners in a particular employment sector. Including
the range and nature of qualifications
assessment
content including currency
comparability
efficiency
Welsh-medium
roles of bodies in the system

3. Sequence of reviews
Health and Social Care (including childcare) – published July 2016
Construction and the Built Environment – to be published Feb 2018
ICT – to be published summer 2018
Engineering and Advanced Construction – to commence Oct 2018
To be followed by:
Financial services
Customer services and retail
Travel and tourism
Hospitality and catering

4. Scale of publicly-fundable Construction-related qualifications in Wales – 2015 certifications

5. Methodology
Involvement of CITB
Stakeholder advisory panel
Interviews and discussion groups – 117+ employers, 70+ others
Learner engagement – focus groups 940 learners
Technical review – 23 qualifications, 14 reviewers
International comparison work – Canada, New Zealand, Australia, Germany
Online survey

6. Model for assessing effectiveness of qualifications
Valid
Reliable
Manageable
Engaging

7. CBE Findings 1 – The qualifications system
System is non-strategic, complex, confusing and proliferated Progression routes neither clear nor sufficient – dead ends, repetition, gaps
Specialisation too early - core, inter-personal and multi-skills are needed
Apprenticeships too short
Challenges in recruiting able learners
Lack of information for learners, employers and parents
Shared apprenticeships add value for learners and employers
Learners on apprenticeships do benefit from learning programmes
Learners feel well supported by their tutors

8. CBE Findings 2 – The qualifications
Inflexible and not responsive to the needs of employers and the wider economy
Content is sometimes outdated, irrelevant and/or insufficient
Poor coverage of skills for traditional buildings
Poor coverage of skills for new building techniques
The balance of theory, practice and real work is not right
Assessment low on validity, reliability, manageability and engagement
Problems with ‘Levels’
Internal and External quality assurance of assessment inconsistent and/or poor
Gaps in the technical knowledge and skills of the assessment workforce
Limited assessment through the medium of Welsh.

9. CBE - Problems with assessment
burden of assessment
language, terminology and format of knowledge assessments
validity of simulated assessment activities
organisation and sufficiency of work-based assessment
availability and validity of naturally occurring evidence
inconsistency of assessment and quality assurance
poor engagement with employers in the assessment process
lack of differentiation in pass/fail qualifications
availability and expertise of assessors
availability of Welsh-medium assessment.

10. CBE - Problems with levels
Significant repetition across levels 1, 2 and 3
Difference in performance between levels unclear/insignificant 
Level of complexity across different trades not equivalent across the ‘same’ level
Real progression within a skills area may be represented by broadening or deepening the learner’s skills and knowledge within a level rather than progression to the next level
Equating level 2 vocational qualifications with GCSEs is unhelpful
Some assumptions about autonomy = supervision 
Mismatch between skills for work and cognitive skills for Higher Education.

11. Problems with Levels
“We are actually assessing vocational students who have chosen a vocational route academically. We are possibly stopping those who could be the best trades people from progression simply because we are judging them on academic values.”
Head of Construction in a Further Education College

12. CBE Proposals for Action – Phase 1
Ask awarding bodies to consider the findings about assessment
Ask sector bodies, with awarding bodies, to review the standards for content
Share findings on apprenticeship programmes with Welsh Government
Share findings with Careers Wales
Take findings into account in addressing reviews of Welsh Bacc and Essential Skills
Consider wider issues about assessment including:
levels in vocational qualifications
the use of technology
the manageability of competence-based assessment

13. CBE - Proposals for action – phase 2
14-16 Level 1 / 2
16-19 progression to HE
Full-time FE
Preparing for Apprenticeships
Apprenticeships
Develop approval criteria and invite submissions from awarding bodies
Is it viable for awarding bodies to amend without a guarantee of being the sole supplier in Wales?
Commission a broad-based Foundation qualification with trade-specific pathways.
Commission a Progression qualification for Construction and for Building Services for those who don’t progress immediately to apprenticeships.
Commission a generic apprenticeship qualification for CBE with:
Pre-registration formative assessment by employer and employer sign-off (not part of qualification)
Work-based project
Professional discussion
Online multi-choice knowledge test

14. Interviews and focus groups Online survey Technical Review
Progress of the ICT Sector Review
Interviews and focus groups
749 learner
63 employers
67 schools
9 colleges
7 universities
Online survey
536 learners
79 learning providers
16 employers
14 others
Technical Review
Complete
Additional work on new qualifications
International Comparison
Comparison of ICT in 11 nations

15. ICT - Feedback from employers
Do not know what the various different qualifications cover
Do not value the IT qualifications taken at – they see these more as ‘user’ qualifications
Learners lack conceptual understanding
Skills/aptitude/experience more highly valued than qualifications
Qualifications don’t give enough practical experience – most completely re-train for specific needs of business
Technically capable candidates often don’t demonstrate necessary softer skills, such as customer service
Learners need to develop problem solving and troubleshooting skills, while understanding processes.
Some employers value vendor-specific qualifications and industry recognised certificates such as Comptia +

16. ICT - Feedback from learners in Further Education
Those taking qualifications for progression to HE are not very engaged with the subject – see them as an ‘easy’ way to get grades for University
Very low satisfaction with currency of content, across ICT qualifications
Very low satisfaction with the hardware and software available
Courses ‘not relevant to the working world’
Far too much ‘writing-up’ for assessments rather than doing practical tasks
No opportunities to be creative
Low confidence in tutors’ knowledge and ability
Frustrations with additional mandatory qualifications

17. ICT - Feedback from teacher events
Pre-occupied with having a GCSE and not a vocational qualification
Questions about the relationship to the curriculum review
High level of resistance to change
Some acceptance that modernisation needed and a wide range of suggestions for new content
Agreement that print-screen approaches to assessment were not engaging
Highly variable learner starting points
Variable teacher currency and flexibility
Support for more assessment of product rather than just of process
Some support for a more project-based approach, but stressed importance of external exams
Benefits of online assessment recognised, but some severe resource constraints; and
Would welcome more engagement with businesses, for example problem-setting, webinars, videos.

18. Everyday users/basic operators
Types of roles
Everyday users/basic operators
Operators / informed users – using corporate systems
Technological Solutions
Interface between users/ operators and maintainers/
creators
Maintainers
Networks, equipment
Creators
Makers
Current provision
Digital/
IT user skills
THE GAP?
Digital solutions
Computer Science
Approach of the Review
Review
Consider Developing – or do the recent new qualifications address this?
Light touch review

19. Summary of underpinning challenges
Variable learner starting points
Variable (and lacking) teacher skills
Variable resources
Wide range of potential content
Pace of change

20. Early thoughts on way forward
Recently introduced qualifications may address some of ‘gap’ issues at 14-16
Project-based approaches to assessment most flexible to address other variabilities
Flexible approaches to content
Identify how to assess digital competence elsewhere
Need creative solutions to address teacher workforce and resources.
To be continued!