1. 5 April 2016
Briefing to the Higher Education Portfolio Committee on review of the draft APPs

2. Reputation promise/mission
The Auditor-General of South Africa has a constitutional mandate and, as the Supreme Audit Institution (SAI) of South Africa, it exists to strengthen our country’s democracy by enabling oversight, accountability and governance in the public sector through auditing, thereby building public confidence.

3. Purpose of the briefing
To provide the Portfolio Committee (PC) with audit insights on the interim review of draft annual performance plans (APPs) of the department and its public entities in order to add value to oversight

4. Key committee considerations when reviewing the APP
Does the APP align with MTSF?
Are indicators relevant and complete?
Is there a logical link between objectives, indicators and targets?
Are the targets realistic i.e. SMART?
Are adequate resources (human and financial) available to achieve APPs?
How can the APP be achieved e.g. Performance contracts/ monitoring
Review quarterly progress against APP targets
Review annual achievement against APP targets
Review and track changes to strategic plans and annual performance reports.
Are the Strategic Plans and Annual Performance Plans aligned to the Medium-Term Strategic Framework (MTSF) outcomes and priorities?
Are indicators relevant to the institution’s mandate and the realisation of strategic goals and objectives?
Does the APP include a complete set of indicators that is relevant to the objectives that need to be achieved e.g. are all customised sector indicators relevant to the institution included in the APP?
Is there a logical link between objectives, indicators and targets?
Are the targets realistic (can it be done)?
Does the institution have adequate resources (human and financial) available to achieve predetermined objectives?
How is the institution ensuring that Strategic Plans and Annual Performance Plans will be achieved (performance contracts, quarterly monitoring, etc.)
The Portfolio committee should review the quarterly progress of institution with regard to predetermined objectives and contributions to the MTSF.
The Portfolio committee should review annual achievement of predetermined objectives by institutions (Annual Reports)
The Portfolio committee should approve and track changes to strategic plans and annual performance reports.

5. AGSA review of the draft 2016-17 APP
Review process
Assessed the process followed by departments to prepare and submit strategic plans (if relevant) and APPs.
Assessed the measurability and relevance of the final draft indicators and targets planned for selected programmes
Reporting
Findings from the review are communicated in the interim management report to enable changes to be made.
Findings relevant to the interim review do not have an impact on the audit conclusion on usefulness or reliability of the selected programmes for the PFMA year end audit.

6. Criteria used to assess the draft APP
Measurability of indicators and targets
Indicators are well-defined
Indicators are verifiable
Targets are specific, measurable and time-bound.
Relevance of indicators and targets
Indicators and targets are relevant to the mandate and realisation of strategic goals and objectives.
1. For indicators to measurable, they must be:
Well-defined i.e. clear, unambiguous definition so that data will be collected consistently and will be easy to understand and use.
Verifiable i.e. it must be possible to validate the processes and systems.
2. For targets to be measurable, they must be:
Specific i.e. the nature and the required level of performance can be clearly identified
Measurable: the required performance can be measured
Time bound: the time period or deadline for delivery is specified
3. For indicators to be relevant i.e. the indicator must relate logically and directly to an aspect of the institution's mandate, and the realisation of strategic goals and objectives.

7. Comparison between 2016/7 and 2015/16 budget
16%
20%
12% 1% 2%
99%
32%
Note: The information contained in this slide must be populated with the programme and budget information relevant to specific auditees.
This include an analysis of the budget, how much will be spent on various programmes, the purpose is to determine the extent to which the allocated resources are geared towards service delivery
2015/16 budget aligned to revised estimates for 2015/16 as per 16/17 draft APP submitted for audit.
Total published adjusted estimates of national expenditure for 2015/16 for DHET is R compared to revised estimate for 2015/16 as per draft 16/17 APP of R
% Budget increase
% Budget decrease % %

8. Budget analysis for 2016/14 – Department (cont.)
Economic classification
Department of Higher Education and Training
% budget spend on employee remuneration
Note: Please populate graph with information relevant to specific auditees.

9. Review findings on draft 2016/17 APP
Department of Higher Education and Training
Scope
We assessed the indicators and targets against the “SMART” criteria for selected programmes.
Outcome
Based on the interim review of the performance indicators and targets over which the department has direct control there were no significant findings identified that relate to the “SMART” criteria.
Technical consultations are currently taking place regarding the audit approach and reporting requirements for the consolidated outcome indicators with systematic targets. .

10. Review findings on draft 2016/17 APP
SETAs and other public entities
Scope
We assessed the indicators and targets against the “SMART” criteria for programmes/objectives directly related to service delivery
Outcome
Significant findings were identified for the following SETAs and other public entities
CHIETA
LGSETA
SASSETA
SERVICES
EWSETA
NSFAS
SAQA

11. No. of objectives tested as per APP % objectives not consistent
Review findings on draft 2016/17 APP
Consistency – Objectives not consistent with the strategic plan
Public entity
No. of objectives tested as per APP
Findings
% objectives not consistent
SAQA * 5 2
40%
* SAQA agreed to make the corrections to the 2016/17 APP and are currently in the process of making these amendments which will be submitted to the board for approval, where after a revised APP will be submitted.
Use slide if interim review was performed and findings were raised.
Measurability of targets
Specific
Measurable
Timebound
Measurability of indicators
Well defined
Verifiable
Relevance
Indicators are relevant to mandate and realisation of goals.

12. No. of indicators tested % indicators not well defined
Review findings on draft 2016/17 APP
Measurability – Indicators not well defined
Public entity
No. of indicators tested
Findings
% indicators not well defined
CHIETA 32 10
31%
NSFAS 8 4
50%
Use slide if interim review was performed and findings were raised.
Measurability of targets
Specific
Measurable
Timebound
Measurability of indicators
Well defined
Verifiable
Relevance
Indicators are relevant to mandate and realisation of goals.

13. No. of indicators tested % indicators not verifiable
Review findings on draft 2016/17 APP
Measurability – Indicators not verifiable
Public entity
No. of indicators tested
Findings
% indicators not verifiable
NSFAS 8 4
50%
Use slide if interim review was performed and findings were raised.
Measurability of targets
Specific
Measurable
Timebound
Measurability of indicators
Well defined
Verifiable
Relevance
Indicators are relevant to mandate and realisation of goals.

14. % of targets not specific
Review findings on draft 2016/17 APP
Measurability – Targets not specific
SETA
No. of targets tested
Findings
% of targets not specific
LGSETA 39
100%
SERVICES 62 58
94%
SASSETA* 17 11
65%
EWSETA 45 20
44%
NSFAS 8 4
50%
SAQA ** 25
32%
* SASSETA is in the process of developing a tracking tool to ensure alignment of planned targets to PIVOTAL skills.
** SAQA agreed to make the corrections to the 2016/17 APP and are currently in the process of making these amendments which will be submitted to the board for approval, where after a revised APP will be submitted.
Use slide if interim review was performed and findings were raised.
Measurability of targets
Specific
Measurable
Timebound
Measurability of indicators
Well defined
Verifiable
Relevance
Indicators are relevant to mandate and realisation of goals.

15. % targets not measurable
Review findings on draft 2016/17 APP
Measurability – Targets not measurable
Public entity
No. of targets tested
Findings
% targets not measurable
NSFAS 8 4
50%
SAQA * 25 12
48%
* SAQA agreed to make the corrections to the 2016/17 APP and are currently in the process of making these amendments which will be submitted to the board for approval, where after a revised APP will be submitted.
Use slide if interim review was performed and findings were raised.
Measurability of targets
Specific
Measurable
Timebound
Measurability of indicators
Well defined
Verifiable
Relevance
Indicators are relevant to mandate and realisation of goals.

16. QUESTIONS