Module 9: Analysing Data: Education Indicators 2 A Training Course for the Analysis and Reporting of Data from Education Management Information Systems (EMIS)

Overview Analysing Data: Education Indicators 2 Data Analysis using indicators Disaggregation of indicators Analyzing Disparities Education For All Goals Pacific Education Development Framework

Data analysis using education indicators Data analysis is the systematic investigation of a set of data to identify patterns and trends by applying standard statistical methods and techniques Comparison of statistics over time provides an indication of change, referred to as statistical indicators Data analysis focuses on education indicators to monitor, evaluate and assess progress towards the EFA goals. The purpose of education indicators is to support the monitoring and evaluation processes of planners and policy analysts Data analysis using education indicators Data analysis is the systematic investigation of a set of data to identify patterns and trends by applying standard statistical methods and techniques. The comparison of statistics over time provides an indication of change, and are commonly referred to as statistical indicators.   Data analysis focuses on education indicators to monitor, evaluate and assess progress towards the EFA goals. Indicators support the monitoring and evaluation processes of educational planners and policy analysts.

Monitoring, Evaluation & Assessment Monitoring is conducted on a continual basis during the implementation of a project or programme to track progress and identify issues with the implementation process; Evaluation is carried out on a periodic basis during and after the completion of a project to assess the extent to which the project objectives have been achieved. Assessment is a process of monitoring and evaluating the educational outcomes achieved as a result of the teaching and learning of the school curriculum. Monitoring is conducted on a continual basis during the implementation of a project or programme to track progress and identify issues with the implementation process; Evaluation is carried out on a periodic basis during and after the completion of a project to assess the extent to which the project objectives have been achieved. Assessment is a process of monitoring and evaluating the educational outcomes achieved as a result of the teaching and learning of the school curriculum.

Procedures for data analysis The basic steps for monitoring are: Assess progress and gaps in the achievement of the national and global targets Identify the remaining gaps in terms of access, quality and equity at the sub-national level Review and identify problems, issues, policies, strategies, actions and success stories Use the results to formulate better policies and strategies for achieving goals Assess progress Identify gaps Review issues Use results Procedures for data analysis   EFA Monitoring The basic steps for monitoring are: 1. Assess progress and gaps in the achievement of the national and global targets. 2. Identify the remaining gaps in terms of access, quality and equity at the sub-national level 3. Review and identify problems, issues, policies, strategies, actions and success stories. 4. Use the results to formulate better policies and strategies for achieving goals

Types of Analysis Education indicators can be analysed according to their: characteristics, distribution and patterns differences, disparities and imbalances changes over time (time-series) progress and shortfalls against targets and plans Types of Analysis Education indicators can be analysed according to their: • characteristics, distribution and patterns • differences, disparities and imbalances • changes over time (time-series) • progress and shortfalls against targets and plans

Types of Data For analytical purposes we can organize data into two types: cross-sectional data time-series data. Cross-sectional data capture a situation at a particular time, often referred to as a snapshot e.g. number of schools, students and teachers by district at the start of the 2015 school year Time series data consist of a series of snapshots showing the changes over a period of time e.g. the number of students who attended school each year between 2010 and 2014 Types of Data For analytical purposes we can organize data into two types: cross-sectional data and time-series data.  Data are often analysed using both cross-sectional and time-series methods. Cross-sectional data capture a situation at a particular time, often referred to as a snapshot ; For example, the number of schools, students and teachers by district at the start of the 2015 school year. Time series data consist of a series of snapshots showing the changes over a period of time; For example, the number of students who attended school each year between 2010 and 2014.

Disaggregation of indicators is the process of breaking down and analysing an indicator by detailed sub-categories can reveal differences and disparities that may be hidden with-in national indicators can allow us to understand the nature of disparities within the national education system Disaggregation of Indicators Disaggregation is the process of breaking down and analysing an indicator by detailed sub-categories. Disaggregation of data can reveal differences and disparities that may be hidden with-in national-level indicators

Use of disaggregated indicators We can use disaggregated education indicators to: find differences in the progress towards achieving national goals by comparing indicators at the school, district, provincial and country level provide information that can be used to develop and implement new policies and actions at the sub-national level Good practice: look at more detailed breakdowns, rather than just the aggregate totals, to see how the situation changes over time. When we compare indicators at the school, district, provincial and country level, we can obtain useful information about differences in the progress toward achieving national goals. This information can then be used to develop and implement new policies and actions at the sub-national level. It is good practice to always look at more detailed breakdowns, rather than just the aggregate totals, to see how the situation changes over time.

Disaggregation of Education Indicators Disaggregation of education indicators by level of education geographical areas population sub-groups School records and school census questionnaires collect data that can be used to disaggregate education indicators, such as: enrolments by gender, age and grade teachers by qualification and training schools by type, authority and region School records and school census questionnaires collect data that can be used to disaggregate indicators, such as: enrolments by gender, age and grade; teachers by qualification and training; and schools by type, authority and region. Analysing data and comparing indicators according to these breakdown classifications can help us to better understand the characteristics, patterns, differences and disparities among individual schools, classes, students and teachers.

Levels of Disaggregation Main levels of disaggregation for geographic locations: District or Island level comparison between schools, students and teachers in the local area, and with other districts and islands. Provincial or State level comparison of districts and islands with those in other provinces or states. National level monitor and identify disparities within the education system for the country as a whole, and compare national trends with other Pacific Island education systems. Levels of disaggregated data At district or island level, comparisons can be made between schools, students and teachers in the local area, and with schools, students and teachers in other districts and islands. At provincial or state level, we can compare districts and islands with those in other provinces or states.At the national level, the Ministry of Education can monitor and identify disparities within the education system for the country as a whole, and compare national trends with other Pacific Island education systems.

Dimensions of disaggregation Education indicators can be further analysed by relevant disaggregated classifications to monitor the equality of access and quality of education for different groups. For example: Enrolment ratios for boys and girls to access gender disparities in participation Pupil-teacher ratios for different classes and grades to improve teacher assignment Percentage of qualified teachers by school or by district to compare education quality Dimensions of disaggregation Disaggregation of indicators is especially important in monitoring and evaluation to ensure equality of access and quality of education. It is important to produce and use education indicators that can be further analysed by relevant disaggregated classifications in order to identify specific concerns. For example: Enrolment ratios for boys and girls to access gender disparities in participation Pupil-teacher ratios for different classes and grades to improve teacher assignment Percentage of qualified teachers by school or by district to compare education quality

Selection of Disaggregated Data Selection of the right type of disaggregation will depend on the policy and planning needs of the Ministry of Education. For example if there are concerns about: low participation in education among children of a specific age, an age-specific enrolment ratio will need to be calculated. teachers lacking practical teaching skills, the percentage of untrained teachers may be relevant if planning to conduct in-service teacher training. Selection of Disaggregated Data Different types of disaggregation may be possible for an indicator, such as enrolment ratios by gender, and also by age and grade. Selection of the right type of disaggregation will depend on the policy and planning needs of the Ministry of Education. For example, if there are concerns about low participation in education among children of a specific age, an age-specific enrolment ratio will need to be calculated. As another example, the percentage of untrained teachers may be more relevant than the percentage distributions of teachers by academic qualification if planning to conduct in-service teacher training.

Types of disaggregation The following types of disaggregation can be included in school records and survey data collection for the purpose of monitoring disparities in education: Gender: girls and boys Geographical areas: regions, islands, location (urban/rural) Administrative units: provinces, districts, localities Social groups: socio-economic status, citizenship, birth-place Ethno-cultural identity: ethnicity, religious affiliations, language Vulnerable groups: children of poor families, migrant families, Disabilities: physical, mental, learning School type: private, public and government, church Types of disaggregation The following types of disaggregation can be systematically included in school records and survey data collection for the purpose of monitoring disparities: • Gender: girls and boys • Geographical areas: regions, islands, location (urban/rural) Administrative units: provinces, districts, localities Social group: socio-economic status, citizenship, birth-place • Ethno-cultural identity: ethnicity, religious affiliations, language group • Vulnerable group: children of poor families, migrant families, working children • Disabilities: physical, mental, learning • School type: private, public and government, church

Advantages of Disaggregation A key advantage of disaggregated indicators is that they can be used in evaluating the outcomes and impact of strategic plans and policies on specific population groups and local areas For example… Outcomes evaluations can use enrolment indicators disaggregated by gender and region to provide evidence of progress of policies to ensure equal access of girls to schools Impact evaluations of governmental policies can be made with the disaggregation of indicators and policies and plans can be made based on valid evaluation results. Advantages of Disaggregation A key advantage of disaggregated indicators is that they are very useful in evaluating the outcomes and impact of strategic plans and policies on specific population groups and local areas. For example, enrolment indicators disaggregated by gender and region provide policy analysts and planners with evidence about the progress of policies to ensure equal access of girls to schools across the country. An impact evaluation of governmental policies can be made with the disaggregation of indicators, and subsequently policies and plans can be made based on valid evaluation results.

Analysing Disparities When comparing indicators that have been calculated for each administrative level, area and population group, we can access the degree of disparities between various groups and identify the disadvantaged population groups and areas. Disparities in: Admin Levels Geographic Areas Population Groups Opportunities National Gender Resources Regional Province Age Access Local District Ethnicity Quality School Island Poor Outcomes Village Disabilities Analysing Disparities When comparing indicators that have been calculated for each administrative level, area and population group, we can access the degree of disparities between various groups and identify the disadvantaged population groups and areas. We need to analyse disparities in the distribution of educational opportunities, resources, access, quality and outcomes across administrative levels, geographic areas and population groups.

Statistics for analysing disparities Statistical methods for measuring the size and frequency of various types of disparities: Charts and graphs can be used to make visual comparisons of different sub-populations Statistics can be used to analyse disparities between different groups: Percentage difference Ratio Range Mean and median Percentile and quartile Statistical Methods There are statistical methods for measuring the size and frequency of various types of disparities. The measurement of disparities can be analysed using basic charts and graphs to make visual comparisons of different sub-populations. In addition, the following measurements can be used to analyse disparities between different groups: • Percentage difference • Ratio (gender and location) • Range • Mean and median • Percentile and quartile

Education for all (EFA) At the World Education Forum in 2000, education policy-makers from all countries together with relevant development agencies, NGOs, institutions and educationists jointly adopted the Dakar Framework for Action which aims at achieving six EFA goals by the year 2015.

EFA goals The EFA goals are about ensuring all persons have full access and opportunity to participate in basic education of good quality, so they can acquire the literacy and life skills they need for a decent living and learning throughout life: Goal 1: Comprehensive Early Childhood Care and Education Goal 2: Universal Primary Education for all children Goal 3: Access to Lifelong Learning and Life Skills Goal 4: Access to Adult Literacy & Continuing Basic Education Goal 5: Gender Equality in primary and secondary education Goal 6: Quality of Education improvements

Key EFA indicators Gross Intake Rate (GIR) % of female enrollments Net Intake Rate (NIR) Gross Enrolment Ratio (GER) Net Enrolment Ratio (NER) Percentage of repeaters Repetition Rate (RR) by grade Promotion Rate (PR) by grade Dropout Rate (DR) by grade Survival Rate to Grade 5 Cohort Completion Rate Transition Rate (TR) Percentage of trained teachers Pupil/Teacher Ratio (PTR) % of female enrollments % of female teachers Gender Parity Index % of teachers with qualifications % of teachers certified to teach Pupil/Class Ratio (PCR) Textbook/Pupil Ratio (TPR) Public expenditure on education % of schools with safe water % of schools with sanitation % of pupils w. basic competency School life expectancy

The Pacific Education Development Framework (PEDF) The PEDF framework is based on: commitments made by Pacific countries to global education calls for action EFA, MDGs, UN Decade of Education for Sustainable Development & UN Literacy Decade national and regional responses to the specific needs and challenges in respect of education in the Pacific region mainstreamed into national education strategies (ESSP) regional development strategies (SPC, PIFS) The Pacific Education Development Framework   The PEDF framework is grounded in two sets of imperatives. First the commitments made by Pacific countries to global education calls for action and second, the national and regional response to the specific needs and challenges in respect of education in the Pacific region. The global commitments include the Education for All (EFA) agenda and goals, the Millennium Development Goals (MDGs) relating directly or indirectly to education, the United Nations Literacy Decade and also the UN Decade of Education for Sustainable Development. At the national level in most countries, these international commitments have been mainstreamed into the sectoral planning process.

PEDF goals The vision of PEDF is the provision of quality education for all in Pacific Island countries and has three strategic goals: 1. To achieve universal and equitable participation and access to Pacific education and training (Access & Equity) 2. To improve quality and outcomes. (Quality) 3. To achieve efficient and effective utilisation of resources ensuring balanced and sustained development of Pacific education systems. (Efficiency & Effectiveness) The vision of PEDF is the provision of quality education for all in Pacific Island countries. The mission is to enable each Pacific learner to develop all his/her talents and creativities to the full and thereby enabling each person to take responsibility for his/her own life and make a meaningful contribution to the social, cultural and economic development of Pacific society. The PEDF has three strategic goals: 1. To achieve universal and equitable participation and access to Pacific education and training (Access & Equity) 2. To improve quality and outcomes. (Quality) 3. To achieve efficient and effective utilisation of resources ensuring balanced and sustained development of Pacific education systems. (Efficiency & Effectiveness)

Key PEDF indicators Net Enrolment Ratio Gross Enrolment Ratio Repetition Rate Drop-Out Rate Promotion Rate Transition Rate % Schools with Access to IT % Out-of-school children returning % Students affected by HIV-AIDS Children with special needs enrol Literacy Rate Numeracy Rate Student Teacher Ratio Student Classroom Ratio Student Textbook Ratio Student Computer Ratio % Teachers using vernacular for tching % Schools with clean water /sanitation % School leavers leaving w. qualification Frequency of curriculum review Integration of ESD into curriculum Integration of HIV-AIDS into curriculum Efficiency Rate (Coefficient of Efficiency) Compulsory Education Policy developed Language Policy developed % Qualified teachers as per standards % Untrained teachers teaching 5+ years % Pre-service teacher recruitments % Pre-service training placements Beginning teacher induction program % teachers have professional development % teachers maintaining registration

Exercise Discuss: Does your country use indicators to monitor and evaluate your education strategic plan? What types of disaggregation of indicators does your country produce? What are the main causes of disparity in your country? What indicators does your country report on for EFA? What indicators does your country report for the PEDF?