1. National Skills Conference 23-24 March 2017 St. Georges Hotel, Irene
    Measuring for impact: implication for the monitoring and evaluation framework for skills development
National Skills Conference
23-24 March 2017
St. Georges Hotel, Irene

2. Background
The White Paper for Post School Education and Training (WP-PSET) has expanded the role of the NSA by stating that ‘a restructured NSA will concentrate specifically on monitoring and evaluating the SETAs’. This implies that it will become an expert body with high- level monitoring and evaluation skills .
The NSA approached GTAC to assist with technical support for the development of a business case and implementation plan for its repositioning including business development model, organogram, MTEF costing.
The M&E framework is one of the first deliverables to be produced

3. IMPORTANT COMPONENTS OF AN M&E FRAMEWORK
Logic Model
Theory of Change
Performance Monitoring Framework
Evaluation Framework
Data Management Framework
Analysis and Reporting Framework

4. PURPOSE OF THE FRAMEWORK
Serve as the overarching plan for undertaking monitoring and evaluation including step-by-step guide to operationalization and application of the monitoring and evaluation activities within SETAs.
Provide NSA stakeholders with guidance and tools for undertaking monitoring and evaluation functions of SETAs over the lifespan of the NSDS.
Define the parameters of routine performance monitoring and evaluation by the SETAs.

5. National Skills Authority Department of Higher Education and Training
THE ROLE OF THE FOLLOWING STAKEHOLDERS ARE CRITICAL IN THE IMPLEMENTATION OF THE M&E FRAMEWORK
National Skills Authority
Department of Higher Education and Training
SETAs
TVET Colleges
Universities
Quality Councils
Business sector
Trade Unions
NGOs
Civil Society Organisations

6. ROLE OF NSA
Oversee the management and implementation of the M&E Framework and to ensure the operational integrity of the M&E system.
Conduct research and engages in critical and analytical reflection as well as issue recommendations to stakeholders for compliance with the framework.
Develop and review guidelines, checklists, templates and other tools used to facilitate the application of the framework.
Develop and promote standards for evaluation and quality assurance of performance information.

7. PERFORMANCE MONITORING FRAMEWORK
Goal
Outputs
Performance Indicator
Frequency of Capturing
Data Source
Increasing access to occupationally-directed programmes
SETAs research and identify middle level skills needs in their sectors and put in place strategies to address them, particularly through the use of the public FET colleges and universities of technology working in partnership with employers providing workplace-based training.
Number of undergraduate learners receiving bursaries
Annually
SETA database
Projects are established to address middle level skills in each sector.
SETAs establish projects and partnerships to enable the relevant number of artisans for their sector to be trained, to qualify and become work ready.
Number of artisans successfully completed training programme
Training Providers Learner database

8. PERFORMANCE MONITORING FRAMEWORK
Goal
Outputs
Performance Indicator
Frequency of Capturing
Data Source
Increasing access to occupationally-directed programmes
Sector skills plans identify the supply challenges in relation to high level scarce skills gaps and set out strategies for addressing them.
Number of post- graduate learners receiving bursaries for critical skills areas
Annually
SETA database
NSFAS database
Agreements are entered into between SETAs, university faculties and other stakeholders on appropriate interventions to support improved entry to priority programmes, increased work experience and experiential learning for students and access to post-graduate work.
Sector skills plans identify the focal areas for research, innovation and development.
Research initiatives undertaken by SETA
Semi-annually
SETA reports
Agreements are entered into between SETAs, university faculties and other stakeholders on flagship research projects linked to sector development in a knowledge economy.
Number of TVET College partnership initiatives by sector
Quarterly
Programmes are put in place that focus on the skills needed to produce research that will be relevant and have an impact on the achievement of economic and skills development goals.
Number of University partnership initiative by sector

9. PERFORMANCE MONITORING FRAMEWORK
Goal
Outputs
Performance Indicator
Frequency of Capturing
Data Source
Encouraging better use of workplace-based skills development
SETA stakeholders agree on the provision of substantial quality programmes for employed workers and report on the impact of the training.
Number of enrolled unemployed learners
Annually
SETA learner databases
Number of enrolled employed learners
Sector projects are put in place to address specific sector skills gaps.
Number of certificated unemployed learners
Stats SA Labour Bulletin
Number of certificated employed learners
SETA database?
Cross-sectoral projects are established to address skills needs along local supply chains aimed at supporting local economic development.
Monthly
NPI reports
Quarterly
STAT SA Economic Outlook Report
Absorption rates of trained people

10. PERFORMANCE MONITORING FRAMEWORK
Goal
Outputs
Performance Indicator
Frequency of Capturing
Data Source
Increasing the public sector capacity for improved service delivery and supporting the building of a developmental state
SETAS with responsibility for public sector training conduct analysis and reflection on
achievements and challenges.
Sector analysis reports submitted
Quarterly
 SETA Reports
Sector skills plans set out the capacity needs of relevant departments and entities.
SSP submitted to SETAs by relevant departments
Annually
Plans and funding arrangements are agreed between the relevant departments/entities and the SETAs, and are reported on.
Rand value of initiatives supported by SETAs to support public sector
Building career and vocational guidance
Career guides are developed with labour market information from SETAs, addressing sub-sectors within their sector.
Number of learners receiving career guidance information
Monthly
Sector stakeholders are engaged and programmes are adjusted to meet the skills and qualification needs to promote comprehensive career development.
Number of stakeholder sessions held to promote career guidance

11. Are the SETAs addressing the needs of the employers and employees?
EVALUATION FRAMEWORK PROVIDES THE BASIS FOR NSA TO ANSWER THE FOLLOWING QUESTIONS:
Have the skills development programmes contributed towards the improvement of on-the-job performance of learners?
Are the SETAs addressing the needs of the employers and employees?
Are the training and learning programmes responsive to the skills demands in the sub-sectors?
Are the programmes generally contributing towards improvement in the business results?
Are the programmes contributing to or supporting the sector’s growth and development as well as related national growth and development priorities?
Are the SETAs addressing the informal economy, SMEs?

12. Vision Impact Statement Impact Indicators Data Sources
EVALUATION FRAMEWORK
Vision
Impact Statement
Impact Indicators
Data Sources
A skilled and capable work that shares in and contributes to the benefits and opportunities of economic expansion and an inclusive growth path
Increase access to training and skills development to achieve fundamental transformation of inequities in our society and to improve productivity in the economy
% output per industry/sector
Stats SA Quarterly Report
Export rate of manufacturing goods
Income levels per sector/industry
Employment rate
Stats SA Quarterly Labour Bulletin
Productivity levels by sector
NPI Reports

13. Outcome/Impact Indicators
EVALUATION FRAMEWORK
Goal
Outcomes
Outcome/Impact Indicators
Data Sources
Increasing access to occupationally- directed programmes
Middle level skills needs are identified and addressed in all sectors
% increase in learners gaining employment in middle level in all sectors
Employer HR Records
Artisans qualify with relevant skills and find employment
Number of artisans employed
High levels national skills needs are being addressed by work ready graduates from higher education institutions
% of graduates in gainful employment in all sectors
 Employer HR Records
Relevant research and development and innovation capacity is developed and innovative research projects established
% increase in Research & Development projects
 Research Reports from Universities
% Increase in registered patents by learners from TVET colleges
 Register of Patents

14. Outcome/Impact Indicators
EVALUATION FRAMEWORK
Goal
Outcomes
Outcome/Impact Indicators
Data Sources
Encouraging better use of workplace-based skills development
Training of employed workers addresses critical skills, enabling improved productivity, economic growth and the ability of the work to adapt to change in the labour market
% improvement in productivity
 Employer Records
NPI Reports
Economic growth rate by sector
% GDP output by sector
 Stat SA Reports
% increase in literacy levels of employees
 Stats SA Reports
% reduction in occupational accidents
OHS Council Reports
Employment equity levels
 EEC reports
Vacancy rates
 Employer HR Records
% increase in income levels
Increasing the public sector capacity for improved service delivery and supporting the building of a developmental state
Education and training plans for the public sector are revised and programmes are implemented to build capacity
DPSA Reports
Building career and vocational guidance
Career paths are mapped to qualifications in all sectors and sub- sectors and communicated effectively, contributing to improved relevance of training and greater mobility and progression
% Increase in learners registering for scarce skills by sectors
 SETA Reports

15. NEXT STEP
Finalise the framework (work on logic model and theory of change)
Develop business process maps for each level of M&E
Develop tools for collecting the data
Agree on reporting formats and cycles
Introduce the framework to SETAs
Collect baseline information for each indicator in the M&E Framework