1. Measuring heterogeneity of the educational system Higher School of Economics, Moscow, 2014 www.hse.ru Alina Ivanova, Fuad Aleskerov, Elena Kardanova, Isak Frumin Higher School of Economics, Moscow, Russia

2. Higher School of Economics, Moscow, 2014 Heterogeneity in education photo  Mass higher education  Freshmen with different social, economic and academic backgrounds  To which extent has the system of higher education become non- homogeneous?  Mass higher education  Freshmen with different social, economic and academic backgrounds  To which extent has the system of higher education become non- homogeneous?

3. Higher School of Economics, Moscow, 2014 Studying heterogeneity in education photo Different views diversification of higher education institutions (Reichert, 2009; Carnevale; Strohl, 2010; Posselt et al., 2012) selectivity of higher education institutions, (Calmand et al., 2009; Hurwitz, 2011; Pastine & Pastine, 2012) heterogeneity of the student population (van Ewijk, 2010; De Paola, Scoppa, 2010; Bielinska-Kwapisz and Brown, 2012) Different methods statistical and econometric tools Standard deviation, coefficient of variation (Murdoch, 2002) Gini coefficient (Bosi, Seegmuller, 2006; Sudhir, Segal, 2008) Multidigraphs (Fedriani and Moyano, 2011) Different views diversification of higher education institutions (Reichert, 2009; Carnevale; Strohl, 2010; Posselt et al., 2012) selectivity of higher education institutions, (Calmand et al., 2009; Hurwitz, 2011; Pastine & Pastine, 2012) heterogeneity of the student population (van Ewijk, 2010; De Paola, Scoppa, 2010; Bielinska-Kwapisz and Brown, 2012) Different methods statistical and econometric tools Standard deviation, coefficient of variation (Murdoch, 2002) Gini coefficient (Bosi, Seegmuller, 2006; Sudhir, Segal, 2008) Multidigraphs (Fedriani and Moyano, 2011)

4. Higher School of Economics, Moscow, 2014 About our work photo  Another approach to estimate heterogeneity in education  Heterogeneity of an educational system  A mathematical model based on the construction of universities’ interval order  The Unified State Examination (USE) scores of Russian students are used to illustrate how our measure of the system’s heterogeneity works..  Another approach to estimate heterogeneity in education  Heterogeneity of an educational system  A mathematical model based on the construction of universities’ interval order  The Unified State Examination (USE) scores of Russian students are used to illustrate how our measure of the system’s heterogeneity works..

5. Higher School of Economics, Moscow, 2014 Our model: Construction of the interval order photo Intervals for 5 universities Graph of the interval order for the 5 universities The incidence matrix

6. Higher School of Economics, Moscow, 2014 Our model: Evaluation of the heterogeneity photo 1.The interval order P is constructed 2.The notion of ideal interval order P id is defined Comparing the matrices for the real and the ideal interval orders and using Formula we calculate the Hamming distance between two interval orders Then as the measure of heterogeneity the Hamming distance is used.

7. Higher School of Economics, Moscow, 2014 Our model: how does it work? photo University Mean USE score Standard deviation Respective intervals Mean - SDMean + SD A6055565 B 56070 C8037783 D9058595 Parameters of “real” universities ABCD A0000 B0000 C1100 D1110 Incidence matrix for ‘real’ data Any ideal system: as an example, clustering For each university - its ideal counterpart with the mean value as the center of the cluster and with the interval [m – SD ; m + SD ] Ideal points by clustering Ideal point leftIdeal point right iA5966 iB5966 iC7892 iD7892 Intervals for clustered data iAiBiCiD iA0000 iB0000 iC1100 iD1100 The incidence matrix for the interval order for clustered data The Hamming distance between real interval order and P I is equal to 0.08.

8. Higher School of Economics, Moscow, 2014 The data The monitoring database "Quality of students’ enrollment - 2012". The database contains; Mean scores of Unified State Examination (USE) of full-time students enrolled in 2012 on the Bachelor Degree Programs, Information on the forms of admission (on competition, out of competition, on a tuition-based basis or on a state-financed basis, etc.). Open data We examine the heterogeneity in the context of integrated groups of majors – in Economics and Management Group of MajorsEconomy and management Total number of universities379 Total number of enrolled students91 336 Score mean (range)46 – 84 Standard deviation (range)3.1 – 17.2

9. Higher School of Economics, Moscow, 2014 The notion of ideal system Simulated data Experts view Artificially ideal Real data Experts view Based on real data

10. Higher School of Economics, Moscow, 2014 Our ideal system Ideal educational system for Economics and Management: expert view 1) A group of best universities that train managers, strategists, high-class analysts (about 10% of universities, the average score of the whole contingent of enrolled students should not fall below 75) 2) A group of strong universities that train strong professionals for regional labor markets (about 70% of universities, the average score from 65 to 74). 3) Group of universities preparing bachelors on applied programs (about 20% of universities, an average USE score of the admitted contingent should not be lower than 55).

11. Interval orders for real universities constructed Real university replaced by “ideal” counterpart Interval orders for “ideal” universities constructed Higher School of Economics, Moscow, 2014 The scores' intervalMeanSt.Dev.Count (%) >75792.8210 (3%) (65;75]693.2952 (14%) (55;65]592.67220 (58%) <=55521.2697 (25%) Prototype for ideal system. Comparing the matrices P for real and ideal interval orders and using formula (2) we can calculate the Hamming distance between two interval orders: H(P, P id )=0.26. The results

12. Higher School of Economics, Moscow, 2014 The desirable lower limit of the average USE scores for economics majors lies at the level of 55 points 97 universities to delete The Hamming distance between real and ideal interval orders becomes H(P,P id )= 0.16. Improving system

13. Higher School of Economics, Moscow, 2014 The Hamming distance between real and ideal interval orders becomes H(P,P id )= 0.21. Improving system: stage 2 Input artificial data Construct ideal orders Compare matrixes GroupsMeanCount (%) Top 10%757528 (10%) Middle 70%68197 (70%) Bottom 20%5057 (20%)

14. Higher School of Economics, Moscow, 2014 To conclude  A new method of studying heterogeneity in the higher education system  Our method is based on the comparison of the hypothetical educational system to the real system  We showed how our method works on Russian data  The model proposed can be applied for any other data, educational systems, countries

15. Higher School of Economics, Moscow, 2014 photo Questions and comments