National Skills Conference 23-24 March 2017 St. Georges Hotel, Irene     Measuring for impact: implication for the monitoring and evaluation framework for skills development National Skills Conference 23-24 March 2017 St. Georges Hotel, Irene

Background The White Paper for Post School Education and Training (WP-PSET) has expanded the role of the NSA by stating that ‘a restructured NSA will concentrate specifically on monitoring and evaluating the SETAs’. This implies that it will become an expert body with high- level monitoring and evaluation skills . The NSA approached GTAC to assist with technical support for the development of a business case and implementation plan for its repositioning including business development model, organogram, MTEF costing. The M&E framework is one of the first deliverables to be produced

IMPORTANT COMPONENTS OF AN M&E FRAMEWORK Logic Model Theory of Change Performance Monitoring Framework Evaluation Framework Data Management Framework Analysis and Reporting Framework

PURPOSE OF THE FRAMEWORK Serve as the overarching plan for undertaking monitoring and evaluation including step-by-step guide to operationalization and application of the monitoring and evaluation activities within SETAs. Provide NSA stakeholders with guidance and tools for undertaking monitoring and evaluation functions of SETAs over the lifespan of the NSDS. Define the parameters of routine performance monitoring and evaluation by the SETAs.

National Skills Authority Department of Higher Education and Training THE ROLE OF THE FOLLOWING STAKEHOLDERS ARE CRITICAL IN THE IMPLEMENTATION OF THE M&E FRAMEWORK National Skills Authority Department of Higher Education and Training SETAs TVET Colleges Universities Quality Councils Business sector Trade Unions NGOs Civil Society Organisations

ROLE OF NSA Oversee the management and implementation of the M&E Framework and to ensure the operational integrity of the M&E system. Conduct research and engages in critical and analytical reflection as well as issue recommendations to stakeholders for compliance with the framework. Develop and review guidelines, checklists, templates and other tools used to facilitate the application of the framework. Develop and promote standards for evaluation and quality assurance of performance information.

PERFORMANCE MONITORING FRAMEWORK Goal Outputs Performance Indicator Frequency of Capturing Data Source Increasing access to occupationally-directed programmes SETAs research and identify middle level skills needs in their sectors and put in place strategies to address them, particularly through the use of the public FET colleges and universities of technology working in partnership with employers providing workplace-based training. Number of undergraduate learners receiving bursaries   Annually SETA database Projects are established to address middle level skills in each sector. SETAs establish projects and partnerships to enable the relevant number of artisans for their sector to be trained, to qualify and become work ready. Number of artisans successfully completed training programme Training Providers Learner database

PERFORMANCE MONITORING FRAMEWORK Goal Outputs Performance Indicator Frequency of Capturing Data Source Increasing access to occupationally-directed programmes Sector skills plans identify the supply challenges in relation to high level scarce skills gaps and set out strategies for addressing them. Number of post- graduate learners receiving bursaries for critical skills areas Annually SETA database   NSFAS database Agreements are entered into between SETAs, university faculties and other stakeholders on appropriate interventions to support improved entry to priority programmes, increased work experience and experiential learning for students and access to post-graduate work. Sector skills plans identify the focal areas for research, innovation and development. Research initiatives undertaken by SETA Semi-annually SETA reports Agreements are entered into between SETAs, university faculties and other stakeholders on flagship research projects linked to sector development in a knowledge economy. Number of TVET College partnership initiatives by sector Quarterly Programmes are put in place that focus on the skills needed to produce research that will be relevant and have an impact on the achievement of economic and skills development goals. Number of University partnership initiative by sector

PERFORMANCE MONITORING FRAMEWORK Goal Outputs Performance Indicator Frequency of Capturing Data Source Encouraging better use of workplace-based skills development SETA stakeholders agree on the provision of substantial quality programmes for employed workers and report on the impact of the training. Number of enrolled unemployed learners Annually SETA learner databases Number of enrolled employed learners Sector projects are put in place to address specific sector skills gaps. Number of certificated unemployed learners Stats SA Labour Bulletin Number of certificated employed learners SETA database? Cross-sectoral projects are established to address skills needs along local supply chains aimed at supporting local economic development.   Monthly NPI reports Quarterly STAT SA Economic Outlook Report Absorption rates of trained people

PERFORMANCE MONITORING FRAMEWORK Goal Outputs Performance Indicator Frequency of Capturing Data Source Increasing the public sector capacity for improved service delivery and supporting the building of a developmental state SETAS with responsibility for public sector training conduct analysis and reflection on achievements and challenges. Sector analysis reports submitted Quarterly  SETA Reports Sector skills plans set out the capacity needs of relevant departments and entities. SSP submitted to SETAs by relevant departments Annually Plans and funding arrangements are agreed between the relevant departments/entities and the SETAs, and are reported on. Rand value of initiatives supported by SETAs to support public sector Building career and vocational guidance Career guides are developed with labour market information from SETAs, addressing sub-sectors within their sector. Number of learners receiving career guidance information Monthly Sector stakeholders are engaged and programmes are adjusted to meet the skills and qualification needs to promote comprehensive career development. Number of stakeholder sessions held to promote career guidance

Are the SETAs addressing the needs of the employers and employees? EVALUATION FRAMEWORK PROVIDES THE BASIS FOR NSA TO ANSWER THE FOLLOWING QUESTIONS: Have the skills development programmes contributed towards the improvement of on-the-job performance of learners? Are the SETAs addressing the needs of the employers and employees? Are the training and learning programmes responsive to the skills demands in the sub-sectors? Are the programmes generally contributing towards improvement in the business results? Are the programmes contributing to or supporting the sector’s growth and development as well as related national growth and development priorities? Are the SETAs addressing the informal economy, SMEs?

Vision Impact Statement Impact Indicators Data Sources EVALUATION FRAMEWORK Vision Impact Statement Impact Indicators Data Sources A skilled and capable work that shares in and contributes to the benefits and opportunities of economic expansion and an inclusive growth path Increase access to training and skills development to achieve fundamental transformation of inequities in our society and to improve productivity in the economy % output per industry/sector Stats SA Quarterly Report Export rate of manufacturing goods Income levels per sector/industry Employment rate Stats SA Quarterly Labour Bulletin Productivity levels by sector NPI Reports

Outcome/Impact Indicators EVALUATION FRAMEWORK Goal Outcomes Outcome/Impact Indicators Data Sources Increasing access to occupationally- directed programmes Middle level skills needs are identified and addressed in all sectors % increase in learners gaining employment in middle level in all sectors Employer HR Records Artisans qualify with relevant skills and find employment Number of artisans employed High levels national skills needs are being addressed by work ready graduates from higher education institutions % of graduates in gainful employment in all sectors  Employer HR Records Relevant research and development and innovation capacity is developed and innovative research projects established % increase in Research & Development projects  Research Reports from Universities % Increase in registered patents by learners from TVET colleges  Register of Patents

Outcome/Impact Indicators EVALUATION FRAMEWORK Goal Outcomes Outcome/Impact Indicators Data Sources Encouraging better use of workplace-based skills development Training of employed workers addresses critical skills, enabling improved productivity, economic growth and the ability of the work to adapt to change in the labour market % improvement in productivity  Employer Records NPI Reports Economic growth rate by sector % GDP output by sector  Stat SA Reports % increase in literacy levels of employees  Stats SA Reports % reduction in occupational accidents OHS Council Reports Employment equity levels  EEC reports Vacancy rates  Employer HR Records % increase in income levels Increasing the public sector capacity for improved service delivery and supporting the building of a developmental state Education and training plans for the public sector are revised and programmes are implemented to build capacity   DPSA Reports Building career and vocational guidance Career paths are mapped to qualifications in all sectors and sub- sectors and communicated effectively, contributing to improved relevance of training and greater mobility and progression % Increase in learners registering for scarce skills by sectors  SETA Reports

NEXT STEP Finalise the framework (work on logic model and theory of change) Develop business process maps for each level of M&E Develop tools for collecting the data Agree on reporting formats and cycles Introduce the framework to SETAs Collect baseline information for each indicator in the M&E Framework