1. 1 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Draft chapter 2 on education output Background document: « chapter 2: education services »

2. 2 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS General links between output and outcome Hill services and result of the production process Peter Hill (1975) says the production of services is inseparable from their use or consumption: two economic units are simultaneously involved in service production whereas production of a good takes place within a single producer unit. Output of services should be viewed as the attributable contribution of the producer to the outcome. Unit ProductProducerConsumer GoodOutput ServiceOutcome

3. 3 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Definitions: education output Eurostat handbook on prices and volume in NA the quantity of teaching received by the students, adjusted to allow for the qualities of the services provided, for each type of education In the area of education, the output can be defined as the quantity of teaching (that is, the transfer of knowledge, successfully or not) for a pupil, whereas the related outcomes are the skill and knowledge that a pupil achieves. ISCED definition organised and sustained communication designed to bring about learning, where Communication involves the transfer of information (messages, ideas, knowledge, strategies, etc.) between two or more persons; […] knowledgeLearning is taken as any change in behaviour, information, knowledge, understanding, attitudes, skills, or capabilities which can be retained and cannot be ascribed to physical growth or to the development of inherited behaviour patterns

4. 4 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Definitions: education output (2) This handbook proposal expected transfer of knowledge and skills provided by an education unit, for each type of level of education / curriculum Transfer of knowledge and skills can be seen as the true quantity indicator, and change in academic scores due to school could provide a right measurement. Expected, we could say « average », means that each pupil receiving the same teaching is supposed to consume the same output, although some pupils will assimilate more than others (what the Eurostat handbook called « successfully or not »). This difference in outcome is supposed due to the pupil/consumer. The ISCED-97 levels of education and secondarily fields of education define the natural strata.

5. 5 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS ISCED-97 levels of education Level 0Pre-primary education Level 1Primary education or first stage of basic education Level 2Lower secondary or second stage of basic education Level 3(Upper) secondary education Level 4Post-secondary non-tertiary education Level 5First stage of tertiary education Level 6Second stage of tertiary education

6. 6 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS ISIC rev. 4 and ISCED 97 ISIC rev 4 classesISCED-97 levels of education 8510Pre-primary and primary education Levels 0 and 1 8521General secondary educationLevels 2 and 3 oriented general 8522Technical and vocational secondary education Levels 2 and 3 oriented vocational and technical 8530Higher educationLevels 4, 5 and 6 8541Sports and recr. education Not classified in ISCED-97 levels of education 8542Cultural education 8549Other education n.e.c. 8550Educational support activitiesNot explicitly mentioned in ISCED-97 levels of education Formal education (our focus) informal Supportactivities But no R&D or adm., like in COFOG or UOE

7. 7 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Basic formula I volume = I quantity (number of enrolled pupils) x I quality ed (average transfer of knowledge) x I quality non ed (total expenditures / core educational) Without information on the transfer of knowledge, we find the number of pupils, 2 nd best quantity indicator for Eurostat. If we assume that the transfer is continuous and proportional to time of teaching, we get the pupil x hours, 1 st best quantity indicator for Eurostat. Non educational quality is theoretically better estimated by an additive formula if detail by secondary product is known.

8. 8 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Consistency between quantity and quality indicators If pupil-hours are the quantity indicator and change in scores the quality indicator, it leads to a double accounting: Imagine a pupil with an attendance of 90% instead of 100%. As he has received 10% less teaching, it is probable that he will perform only 90% of the change in score of his normal schoolmates. If we cumulate the quantity effect (pupil hours) and the quality indicator (change in scores), we obtain 81% instead of 90%, the expected transfer of knowledge for this pupil. transfer of knowledge = number of pupils getting 1 year older * average transfer of 1 year = number of pupils moving up 1 year * average transfer of 1 grade If the quality indicator relies on scores at a given age, the quantity indicator should be the simple number of enrolled pupils. If it relies on scores at a given grade, the quantity indicator should be the number of pupils moving up.

9. 9 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Overview of inputs, output, outcomes in volume

10. 10 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Education output = average transfer of knowledge and skills, some remarks No accounting of the distribution of individual levels of knowledge and skills, although reduction of inequalities is usually one of the targets of the education policy: consistent with the idea of individual services and common rules in NA for FC, income, etc. No accounting of healthy, wealthy, well-rounded people / citizens, which are sometimes difficult to measure and which are more indirect outcomes. We measure usually education status (outcome) at the end of a curriculum, rarely at the beginning, we need therefore a model (part of the pupil / consumer, time-lag) in order to estimate the part of the output in the outcome.

11. 11 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS The model: education output and outcome

12. 12 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Size of the SD: does it matter for NA? The SD of the distribution of education levels can be standardized to 10% or 20% of the average PISA, PIRLS, TIMSS standardized to 20%, US NAEP to 10%. The human capital theory (relying on future real earnings) suggests 8-10% (2 studies) or 20% (1 study). But it does not matter in a NA view The convention for the relative size of 1 SD has an impact both on cross-section (better school vs average school) and longitudinal distances (between ages 6 and 15 for instance, which will provide a measurement for average schools). The quality adjustment in equivalent number of years of teaching is a ratio between cross-section and longitudinal distances.

13. 13 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Relative scores across ages The larger the SD at age 15, the larger the output between ages 6 and 15, and constant proportions between numerator and denominator of QA.

14. 14 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS => For PPP, let us scale the average outcome at age 15 = 500 and 1 SD = 20% (PISA) PISA measures a national average outcome at age 15. To reflect the output of the national school systems, it should be neutralized of the family contribution (ESCS correction) and subtracted of inherited skills. The formula should be: ESCS corrected PISA – X.

15. 15 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Estimation of what is to be subtracted to ESCS corrected PISA scores: at least 180 Ideally, we should observe inherited skills at age 15, or more exactly inherited skills + average family contribution (for ESCS only neutralizes the differences in family contribution) at age 15. Requires a sophisticated model. But we can assume that education status at age 6 reflects only inherited skills + average family contribution. Can be extrapolated with the help of 2 national longitudinal tests: the US NAEP and the UK 4 fey stages, which suggest about 180 at age 6 in average (with a convention of 1 SD = 20%). Inherited skills + average family contribution can increase between ages 6 and 15, so that we could subtract more.

16. 16 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Age US NAEP (1 SD = 10%) UK 4 key stages (1 SD ~ 20%) re-scaled US (age 15 = 500, 1 SD = 20%) re-scaled UK (age 15 = 500, 1 SD = 20%) re-scaled US (age 15 = 500, 1 SD = 10%) re-scaled UK (age 15 = 500, 1 SD = 10%) 6(203)(12.52)182180352340 715.51217223368362 8253267385383 9231288310401405 10323353418426 1127.49359396434448 12394426451463 13429456467478 1433.79465487484493 15(288)(34.71)500 1635.63535513516507 17307571527533513 impact of 1 SD on output (age 15 - 6) 34%31% 34%31%

17. 17 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Time-lag management grade \ year of outcome 2000200120022003 9 th grade (14- 15 year old) PISA 2000 8 th grade (13- 14 year old) 2/3 PISA 2000 + 1/3 PISA 2003 7 th grade (12- 13 year old) 1/3 PISA 2000 + 2/3 PISA 2003 6 th grade (11- 12 year old) PISA 2003 Total 2000: 2 PISA 2000 + 2 PISA 2003 Total 2001: PISA 2000 + 8/3 PISA 2003 + 1/3 PISA 2006 Lower secondary education

18. 18 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Time-lag management and revisions YearsLower secondary (hypothesis of 4 years) Primary (hypothesis of 5 years) 2000 PISA 2000 then ½ PISA 2000 + ½ PISA 2003 PISA 2000 then PISA 2003 then 1/5 PISA 2003 + 4/5 PISA 2006 at last 1/5 PISA 2003 + 3/5 PISA 2006 + 1/5 PISA 2009 2001 PISA 2000 then 1/4 PISA 2000 + 3/4 PISA 2003 at last 1/4 PISA 2000 + 2/3 PISA 2003 + 1/12 PISA 2006 PISA 2000 then PISA 2003 then 1/15 PISA 2003 + 14/15 PISA 2006 at last 1/15 PISA 2003 + 8/15 PISA 2006 + 6/15 PISA 2009 2002 PISA 2000 then 1/12 PISA 2000 + 11/12 PISA 2003 at last 1/12* PISA 2000 + 2/3* PISA 2003 + 1/4* PISA 2006 PISA 2000 then PISA 2003 then PISA 2006 then 6/15 PISA 2006 + 9/15 PISA 2009 at last 6/15 PISA 2006 + 8/15 PISA 2009 + 1/15 PISA 2012 2003 PISA 2003 then ½ PISA 2003 + ½ PISA 2006 PISA 2003 then PISA 2006 then 1/5 PISA 2006 + 4/5 PISA 2009 at last 1/5 PISA 2006 + 3/5 PISA 2009 + 1/5 PISA 2012

19. 19 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Other outcome-based quality indicators Although we think that academic scores are the first best direct education outcomes, some curricula could be better estimated (or are more traditionally estimated) by obtainment of a degree or professional insertion (this latter an indirect outcome). Professional insertion: employment rate (in the right qualification if possible) and real earnings can be used (multiplied) and are often suggested by human capital theory (discounted sum of future real earnings as return on GFCF in education), but they are to be considered in an incremental point of view: what is obtained after the curriculum minus what would have been obtained without, if we want to measure the output.

20. 20 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Which output/outcome for tertiary education? Degrees are certainly the most natural outcome indicators for tertiary education, but they are of different values. Harmonization of degrees in Europe with the Bologna process (bachelor, master, doctorate), but what are the relative values of these 3 degrees? Distinguish by (ISCED-97) field of education; Double degree for one curriculum: double outcome? Curriculum shared between different countries/units; The output should not be null for students leaving without degree. Time-lag adjustment. Number of credits reflect perhaps more directly the quantity of transfer of knowledge and skills, i.e. our definition of the output, and avoid most problems quoted above. Still need to distinguish by field of education and perhaps by prestige of university.

21. 21 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS If not outcome-based then outcome-oriented If the output is defined as the attributable contribution of the producer to the outcome, and if the outcome cannot be directly measured, then some inputs-based or process- based quality indicators could be used, but outcome- oriented, it means converted in an expectation of incremental contribution to the outcome. Example of the Italian model of the class size (see item 3.3). If the more classical way of treating different qualities as separate goods was chosen, it would lead to a quality presentation but not to a quality adjustment: the price index would be nothing else than a cost weighted combination of cost indices, i.e. the same price index as in the input method.

22. 22 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Stratum for time-seriesQuantityQuality (ed)Comment Pre-primary education Pupil-hoursNone Primary education: normal Pupils Contribution to scores The sub-stratification of ISCED levels could be replaced by an estimate of the non- educational part (social services) Primary education: special Lower secondary: normal Lower secondary: special Upper secondary education: general + pre-technical or pre-vocational Do not forget to subtract the entry education status Upper secondary education: vocational Employment rate after 1 year and/or real earnings if no scores available In an incremental point of view and net of the economic trend Post-secondary non-tertiary education More occup. specific prog. tertiary education (5B) More theoretically based programmes tertiary education (5A+6) Credits 1 st best Degrees 2 nd best Pupils 3 rd best by field of ed. value of degrees?

23. 23 Paris 06/06/2007 2 nd OECD workshop on measuring Education and Health volume output Alain GALLAIS, OECD/STD/NAFS Stratum for cross-countryQuantityEd. qualityNon ed. quality ISCED 0: Pre-primary education Pupils (pupil-hours?) NoneShould come in a future step from the UOE data collection, based on the ancillary services, after reconciliation with NA. ISCED 1: Primary education PupilsESCS corrected average PISA score - 180 ISCED 2: Lower secondary: normal Pupils ISCED 3: Upper secondary education Pupils ISCED 4: Post-secondary non-tertiary education PupilsNone ISCED 5B: More practical and occupationally specific programmes tertiary education PupilsNone at present ISCED 5A and 6: More theoretically based programmes tertiary education Pupils (ECTS?) (degrees?) None at present (expectation of degrees?)