1 Kenneth N. Ross University of Melbourne, Australia 2 September 2010 The Development and Application of the SACMEQ HIV-AIDS Knowledge Test for Pupils at the Upper-Primary School Level in 15 African Countries

2 1.Sub-Saharan Africa, HIV-AIDS, and the SACMEQ III Project

3 3

4 To deliver training and research programmes for African educational planners and researchers that will enable them to:  (a) monitor and evaluate the quality of their basic education systems; and  (b) generate research-based information that can be used by decision-makers to plan improvements in the quality of education. SACMEQ’s Mission

5 Context Information for SACMEQ School Systems

6 6 SACMEQ Assembly of Ministers of Education SACMEQ Managing Committee General Policy SACMEQ Co- ordinating Centre Detailed Policy and Monitoring Scientific & Measurement Committees Training and Technical Support “External Friends” Co-operation 15 SACMEQ NRCs and 30 D/NRCs Training and Technical Support General Administration and Country-Specific Technical Issues

7 SACMEQ Conceptual Model 7 School Characteristics: Type, Location, Enrolment, Resources, Principal’s Qualification, Parental Involvement, etc. Learner Characteristics: Age, Sex, Attendance, Repetition, SES, Nutrition, Siblings, Home Help, etc. Teacher Characteristics: Age, Sex, Qualifications, Subjects, Classroom resources, Behaviour, In-Service Training, etc. Learner Achievement Reading Maths HIV/AIDS

8 The SACMEQ I Project The SACMEQ II Project The SACMEQ III Project MoE 20,000 Grade 6 pupils 1,000 schools 14 MoE 40,000 Grade 6 pupils 2,000 schools 15 MoE 61,421 Grade 6 pupils 8,045 teachers 2,779 schools Questionnaires Reading Test for Pupils. Questionnaires Reading & Maths Tests for Pupils and Teachers Questionnaires Reading, Maths & HIV/AIDS Knowledge Tests for Pupils and Teachers

9 Adults and children estimated to be living with HIV (UNAIDS/WHO, 2007)

10 Children (< 15 years) estimated to be living with HIV (UNAIDS/WHO, Dec 2007)

11 Adult HIV/AIDS Prevalence Rate (Aged 15-49) in SACMEQ Countries (UNAIDS, 2007) 1Swaziland26.1% 2Botswana23.9% 3Lesotho23.2% 4South Africa18.1% 5Namibia15.3% 5Zimbabwe15.3% 6Zambia15.2% 7Mozambique12.5% 8Malawi11.9% 9Kenya % 11Tanzania (United Rep. of)6.2% 13Uganda5.4% 29Mauritius1.7% 50SeychellesNA

12 2. SACMEQ HIV-AIDS Knowledge Test (Known as the HAKT)

13 UNGASS Indicator 13: “Knowledge about HIV Prevention” Percentage of young people aged who provide correct answers to the following questions about (a) sexual transmission of HIV, and (b) misconceptions about HIV. SEXUAL TRANSMISSION OF HIV 1. Can the risk of HIV transmission be reduced by having sex with only one uninfected partner who has no other partners? 2. Can a person reduce the risk of getting HIV by using a condom every time they have sex? MISCONCEPTIONS ABOUT HIV 3. Can a healthy-looking person have HIV? 4. Can a person get HIV from mosquito bites? 5. Can a person get HIV by sharing food with someone who is infected?

14 Problems with the UNGASS 13 Indicator 1. LOW RELIABILITY (0.28) 2. LOW CONTENT VALIDITY (a) Poor Curriculum Coverage, and (b) Concentration on “Misconceptions” 3. LINGUISTIC COMPLEXITY (Intersecting questions & qualifiers). 4. INVITATION TO CHANGE INDICATOR QUESTIONS “The first three questions should not be altered. However, Questions 4 & 5 ask about local misconceptions and may be replaced by the most common misconceptions in your country.” 5. WRONG TARGET GROUP (15-24 years is too late). 6. JUSTIFICATIONS GIVEN FOR ITEM SELECTION For example; “The belief that HIV is transmitted through mosquito bites can weaken motivation to adopt safer sexual behaviour.”

15 TWO DEFINITIONS OF AIDS “AIDS, the acquired immunodeficiency syndrome, is a fatal disease caused by HIV, the human immunodeficiency virus.” (UNAIDS, 2007, p. 8). “AIDS is not a disease. It is a syndrome or group of illnesses resulting from a weakening of the immune system.” (UNESCO, p. 28).

16 SACMEQ HIV-AIDS Knowledge Test Based on Official Curriculum, Teaching Guides & Materials Target Group = Grade 6 Pupils (Average Age = 13.5 years) Extensive Approval Process: MoEs (at all levels), Ministers of Education), SACMEQ Scientific Committee, External Experts. Standardized Test Administration (Scripted Group Interview) Balance of True/False Items (86) Administered to over 60,000 pupils in around 2800 schools across 15 countries. Reliability Coefficient = 0.85

17 Administration of the HAKT 1. Two data collectors manage group administration of the HAKT for samples of 25 pupils per school. 2. Extensive training of data collectors via role play. 3. Practice questions (eg. “Elephants can fly like birds.”) 4. Presentation (double reading) of test items. 5. Supervision and checking of progress. 6. After the testing session – explanation/discussion.

18 SACMEQ Knowledge Topics for HAKT

19 Selected Items from the HAKT

Three Pupil & Teacher Performance Scores: Transformed Scores – scaled HAKT scores transformed to a SACMEQ pupil mean of 500 and standard deviation of 100. Minimal Knowledge Scores – dichotomous scores that indicated whether or not respondents had mastered at least 50% of the official curriculum. (Expectation=100% of pupils reach this level) Desirable Knowledge Scores – dichotomous scores that indicated whether or not respondents had mastered at least 75% of the official curriculum. (Expectation=at least half of the pupils reach this level)

21 What knowledge do Grade 6 pupils and their teachers have about HIV- AIDS? Question 1

22

23

24 What are the within-country regional variations in pupil knowledge about HIV-AIDS? Question 2

25 Mozambique Sofala Maputo City Gaza Inhambane

26

27 HAKT Scores by regions

28 Are there gender differences among Grade 6 pupils in knowledge about HIV-AIDS? Question 3

29

30 What was the variation in Grade 6 pupil knowledge about HIV-AIDS – between and within schools? Question 4

31

32 How did Grade 6 teachers rate their own risk of becoming infected with HIV? Question 5

33 Teacher Assessment of Personal Risk of Infection with HIV

34 What percentage of Grade 6 pupils attended lessons on HIV-AIDS? Question 6

35

36 What did Grade 6 pupils nominate as “the best source of information about HIV and AIDS”? Question 7

Best Source of Information on HIV-AIDSPercent Medium Electronic (Radio, TV, Video)42.3 Performance (Play, Drama, Concert) 7.7 Printed (Books, Magazines, Newpapers) 6.8 Computer/Internet 1.7 Person Teacher 17.7 Health Professional 13.0 Family 4.4 Friend or Peer 3.8 Person with HIV 1.6 Religious 0.9 TOTAL100.0

38 What were the “Curriculum Gaps”? Question 8

List of the 25 best understood questions (pupils) (pupils

List of the 25 worst understood questions (pupils)

List of the 25 best understood questions (teacher)teacher)

List of the 25 worst understood questions (teachers)

43 What were the country rankings on the HAKT compared with the UNGASS 13 indicator? Question 9

Comparison of National Rankings on UNGASS and HAKT Indicators

45 FUTURE DIRECTIONS IIEP Policy brochures (2010) SACMEQ III Data Archive ( ) UBW research programme ( ): designing and validating different language versions of the HAKT SACMEQ IV Project FURTHER INFORMATION Kenneth N. Ross University of Melbourne