5 April 2016 Briefing to the Higher Education Portfolio Committee on review of the draft 2016-17 APPs

Reputation promise/mission The Auditor-General of South Africa has a constitutional mandate and, as the Supreme Audit Institution (SAI) of South Africa, it exists to strengthen our country’s democracy by enabling oversight, accountability and governance in the public sector through auditing, thereby building public confidence.

Purpose of the briefing To provide the Portfolio Committee (PC) with audit insights on the interim review of draft annual performance plans (APPs) of the department and its public entities in order to add value to oversight

Key committee considerations when reviewing the APP Does the APP align with MTSF? Are indicators relevant and complete? Is there a logical link between objectives, indicators and targets? Are the targets realistic i.e. SMART? Are adequate resources (human and financial) available to achieve APPs? How can the APP be achieved e.g. Performance contracts/ monitoring Review quarterly progress against APP targets Review annual achievement against APP targets Review and track changes to strategic plans and annual performance reports. Are the Strategic Plans and Annual Performance Plans aligned to the Medium-Term Strategic Framework (MTSF) outcomes and priorities? Are indicators relevant to the institution’s mandate and the realisation of strategic goals and objectives? Does the APP include a complete set of indicators that is relevant to the objectives that need to be achieved e.g. are all customised sector indicators relevant to the institution included in the APP? Is there a logical link between objectives, indicators and targets? Are the targets realistic (can it be done)? Does the institution have adequate resources (human and financial) available to achieve predetermined objectives? How is the institution ensuring that Strategic Plans and Annual Performance Plans will be achieved (performance contracts, quarterly monitoring, etc.) The Portfolio committee should review the quarterly progress of institution with regard to predetermined objectives and contributions to the MTSF. The Portfolio committee should review annual achievement of predetermined objectives by institutions (Annual Reports) The Portfolio committee should approve and track changes to strategic plans and annual performance reports.

AGSA review of the draft 2016-17 APP Review process Assessed the process followed by departments to prepare and submit strategic plans (if relevant) and APPs. Assessed the measurability and relevance of the final draft indicators and targets planned for selected programmes Reporting Findings from the review are communicated in the 2015-16 interim management report to enable changes to be made. Findings relevant to the interim review do not have an impact on the audit conclusion on usefulness or reliability of the selected programmes for the PFMA 2015-16 year end audit.

Criteria used to assess the draft APP Measurability of indicators and targets Indicators are well-defined Indicators are verifiable Targets are specific, measurable and time-bound. Relevance of indicators and targets Indicators and targets are relevant to the mandate and realisation of strategic goals and objectives. 1. For indicators to measurable, they must be: Well-defined i.e. clear, unambiguous definition so that data will be collected consistently and will be easy to understand and use. Verifiable i.e. it must be possible to validate the processes and systems. 2. For targets to be measurable, they must be: Specific i.e. the nature and the required level of performance can be clearly identified Measurable: the required performance can be measured Time bound: the time period or deadline for delivery is specified 3. For indicators to be relevant i.e. the indicator must relate logically and directly to an aspect of the institution's mandate, and the realisation of strategic goals and objectives.

Comparison between 2016/7 and 2015/16 budget 16% 20% 12% 1% 2% 99% 32% Note: The information contained in this slide must be populated with the programme and budget information relevant to specific auditees. This include an analysis of the budget, how much will be spent on various programmes, the purpose is to determine the extent to which the allocated resources are geared towards service delivery 2015/16 budget aligned to revised estimates for 2015/16 as per 16/17 draft APP submitted for audit. Total published adjusted estimates of national expenditure for 2015/16 for DHET is R57 680 138 000 compared to revised estimate for 2015/16 as per draft 16/17 APP of R57 592 643. % Budget increase % Budget decrease % %

Budget analysis for 2016/14 – Department (cont.) Economic classification Department of Higher Education and Training % budget spend on employee remuneration Note: Please populate graph with information relevant to specific auditees.

Review findings on draft 2016/17 APP Department of Higher Education and Training Scope We assessed the indicators and targets against the “SMART” criteria for selected programmes. Outcome Based on the interim review of the performance indicators and targets over which the department has direct control there were no significant findings identified that relate to the “SMART” criteria. Technical consultations are currently taking place regarding the audit approach and reporting requirements for the consolidated outcome indicators with systematic targets. .

Review findings on draft 2016/17 APP SETAs and other public entities Scope We assessed the indicators and targets against the “SMART” criteria for programmes/objectives directly related to service delivery Outcome Significant findings were identified for the following SETAs and other public entities CHIETA LGSETA SASSETA SERVICES EWSETA NSFAS SAQA

No. of objectives tested as per APP % objectives not consistent Review findings on draft 2016/17 APP Consistency – Objectives not consistent with the strategic plan Public entity No. of objectives tested as per APP Findings % objectives not consistent SAQA * 5 2 40% * SAQA agreed to make the corrections to the 2016/17 APP and are currently in the process of making these amendments which will be submitted to the board for approval, where after a revised APP will be submitted. Use slide if interim review was performed and findings were raised. Measurability of targets Specific Measurable Timebound Measurability of indicators Well defined Verifiable Relevance Indicators are relevant to mandate and realisation of goals.

No. of indicators tested % indicators not well defined Review findings on draft 2016/17 APP Measurability – Indicators not well defined Public entity No. of indicators tested Findings % indicators not well defined CHIETA 32 10 31% NSFAS 8 4 50% Use slide if interim review was performed and findings were raised. Measurability of targets Specific Measurable Timebound Measurability of indicators Well defined Verifiable Relevance Indicators are relevant to mandate and realisation of goals.

No. of indicators tested % indicators not verifiable Review findings on draft 2016/17 APP Measurability – Indicators not verifiable Public entity No. of indicators tested Findings % indicators not verifiable NSFAS 8 4 50% Use slide if interim review was performed and findings were raised. Measurability of targets Specific Measurable Timebound Measurability of indicators Well defined Verifiable Relevance Indicators are relevant to mandate and realisation of goals.

% of targets not specific Review findings on draft 2016/17 APP Measurability – Targets not specific SETA No. of targets tested Findings % of targets not specific LGSETA 39 100% SERVICES 62 58 94% SASSETA* 17 11 65% EWSETA 45 20 44% NSFAS 8 4 50% SAQA ** 25 32% * SASSETA is in the process of developing a tracking tool to ensure alignment of planned targets to PIVOTAL skills. ** SAQA agreed to make the corrections to the 2016/17 APP and are currently in the process of making these amendments which will be submitted to the board for approval, where after a revised APP will be submitted. Use slide if interim review was performed and findings were raised. Measurability of targets Specific Measurable Timebound Measurability of indicators Well defined Verifiable Relevance Indicators are relevant to mandate and realisation of goals.

% targets not measurable Review findings on draft 2016/17 APP Measurability – Targets not measurable Public entity No. of targets tested Findings % targets not measurable NSFAS 8 4 50% SAQA * 25 12 48% * SAQA agreed to make the corrections to the 2016/17 APP and are currently in the process of making these amendments which will be submitted to the board for approval, where after a revised APP will be submitted. Use slide if interim review was performed and findings were raised. Measurability of targets Specific Measurable Timebound Measurability of indicators Well defined Verifiable Relevance Indicators are relevant to mandate and realisation of goals.

QUESTIONS