1. A Quantitative Investigation of SBA and Examination Scores in Adult Education in South Africa
13th SAAEA Conference
Gaborone, Botswana
Dr Nkoloyakhe Mpanza
19-22 May 2019

2. Presentation Outline Introduction and Background
Purpose and Research questions
Formulation of hypotheses
Methodology
Findings
Conclusion

3. Introduction and Background
Umalusi is responsible for the qualifications on level 1- 4 of the National Qualification Framework (NQF).
The NQF Level 1 qualification for adults:
provides second chance to adults and
out of school youth who cannot be accommodated in the mainstream schooling sector.

4. Introduction and Background
NQF Level
ABET/Grade Level
5 - 10
HET 4
FET
Grade 12 3
Grade 11 2
Grade 10 1
GET
ABET Level 4 – Grade 9
ABET Level 3 – Grade 7
ABET Level 2 – Grade 5
ABET Level 1 – Grade 3
ABET Level 0 – Pre ABET

5. Introduction and Background
Assessment in this qualification comprises Site-Based Assessment (SBA) and examination.
The SBA and examination contribute 50% each towards the final mark.
SBA scores are included in some of the high stake qualifications around the world (Williamson, 2016).

6. Introduction and Background
Literature revealed inaccuracies regarding SBA in some countries Tailor (2005, in Williamson, 2016),
statistical moderation generally adjusted SBA marks downward in the UK.
van der Berg & Shepherd (2008) identified two types of assessment inaccuracies
Assessment leniency, where SBA mark is far higher than an examination mark; and
Low assessment reliability, where SBA marks and examination marks are weakly correlated.

7. Introduction and Background
For Umalusi:
SBA challenges also observed by Umalusi
Statistical moderation used to validate credibility
Difference above 10% between the means of SBA and that of examination indicate that the SBA is a poor signal of performance.
Hence is considered misalignment
Rejection of SBA scores

8. Purpose of the study & Research question
The study was conducted to investigate whether the means of SBA and examination scores per learning areas that are assessed by the Department of Higher Education and Training (DHET) are comparable.
Research Question
Is the mean of SBA scores comparable to that of examination scores in each of the 5 learning areas assessed the DHET for the NQF Level 1 qualification for adults?

9. Formulated Hypothesis
There is no significant difference between the mean of SBA scores and that of examination scores in the five learning areas assessed by the DHET.

10. Methodology
This quantitative study was conducted using five purposefully sampled learning areas assessed by DHET.
These learning areas are Small Medium and Micro Enterprises (SMME); Mathematical Literacy (MLMS); Mathematics and Mathematical Sciences (MMSC); Economic and Management Sciences (EMSC) and Natural Sciences (NATS).
Source of data: November 2018 examinations and SBA submitted DHET to Umalusi.

11. Methodology Data was analysed using Statistical Analysis System (SAS).
Two-sample t-tests were conducted per learning area to test the formulated hypothesis.
The selected level of significance (alpha) is p= 0.05

12. Findings
Learning Area
Mean-SBA
Mean-Exam
Difference Between Means
Degree of Freedom (df)
P-Value
SMME
59.175
33.357
25.189
7647
NATS
61.926
31.733
30.193
4415
MLMS
60.756
35.476
25.280
38014
MMSC
62.287
38.455
23.832
5359
EMS
62.223
26.166
36.057
10353

13. Findings
Hypothesis
“There is no significant difference between the mean of SBA and that of examination scores in the five learning areas assessed by the DHET.”
Rejection rule:
p-value < alpha then we reject.
The p-value is less than alpha (0.05) in all 5 learning areas, therefore, hypothesis was rejected.
There is a significant difference between the SBA and examination scores in all 5 learning areas

14. Conclusion The study concludes that:
student performance in SBA and examination of selected learning areas is not comparable.
This motivates for a continuation of statistical moderation process.

15. Thank You