1. EVALUATING THE QUALITY OF EARLY CHILDHOOD EDUCATION PROGRAMS Robert Myers Presentation at CIES February 27, 2007

2. ORGANIZATION OF THE PRESENTATION Edging up to an operational definition of educational quality for preschools A Scale for Evaluating the Quality of Preschool Centers Results of the application of the scale in an evaluation of the Mexican Schools of Quality Program Some final reflections

3. WAYS TO APPROACH A WORKING DEFINITION OF EDUCATIONAL QUALITY

4. CONTRASTING FRAMEWORKS Quality as universal, uniform, constant, objetive, tidy (inherent and discoverable quality) or relative, diverse, changing, subjetive, disorderly (negotiated quality)

5. The Intersectorial Project (Negotiating a definition of quality) Participants in the negotiation from: education, health, family welfare, NGOs university Purpose: to define indicadors de well being for children under six Working groups on: –The condition of children – Family and community contexts – Early education programs: their quality

6. A review of research and evaluations Research –What conditions and processes most help children learn? –How should one organize and administer an institution? Evaluaciones : Cuatro dimensiones: Resources The educational process Management Relationships with parents and communithy

7. ASKING MEMBERS OF THE EDUCATIONAL COMMUNITY Personal experience of participants in the project (experts?) Interviews with educational authorities Interviews with directors and educadors about what should be done to improve learning Results: –Varied views, with emphasis on classrooms and on the interactions between the teacher and the child as well as emphasis on improving the resource base. –Context is important

8. THE VIEW OF THE SOCIETY IN WHICH WE WANT TO LIVE Characteristics of the society –Democratic, multi-cultural, respectful of rights, equitable, productive,…… What type of citizen does that require? What type of child should we be educating? –Literate, informed, participatory, critical, tolerant, empathetic, …… ¿What type of education, school and educator is required?

9. The Nacional Education Program, 2001-2006, in Mexico The educational system: Equitable, flexible, descentralized, democratic in its modes of operation, accountable Schools: Offering equal educational opportunity to all, taking into account cultural variations, colegial, open to participation by families and community members, capable of evaluing and communicating results Type of educador: Technically competent, capable of motivating and facilitating active learning and development, participatory, respectful of differences, egalitarian, colegial, able to work with parents as well as children, ….

10. Establish criteria –Active learning: The child constructs knowledge –Integral: in the definition of development and learning applied –Relevant: to the life/interests of the child –Pertinent: within a multi-cultural environment –Participativa: including families/community –Flexible: adjusts to differences in children and contexts (María Victoria Peralta, 2000)

11. A SCALE FOR EVALUATING THE QUALITY OF PRESCHOOL CENTERS

12. THE CHALLENGE Convert the foregoing into indicators of educational quality in order to evaluate and monitor quality. Develop a methodology and instruments that can provide the information needed to utilize the indicators.

13. First Steps Establish dimensions of quality Within each dimension, identify indicators, Examine methods and instruments in use (in Mexico and in other countries) Construct questions and guides for systematic collection of information through conversations and observations

14. DECISIONS Four main dimensions (at first): resources, management, the educational process, relation to families and community Center and classrooms Utilize a continuum (instead of yes-no options) de 5 categorías: –1 = inadecuade –2 = incipient –3 = básic- a kind of minimum all centers should be able to obtain on each indicator –4 = good –5 = excellent

15. Example of a scale item: The educational process Indicator: The process fosters and permits children to chose topics, activities and materials and to put in practice their own ideas Inadecuate 1 The teacher: - chooses the theme, activities and materials. - does not accept suggestions or variations Incipiente 2 The teacher: - chooses the theme, activities and materials - sometimes accepts suggestions or variations Básic 3 There are moments during the day when the teacher proposes the theme and the children choose the activity and/or materials Bueno 4 There are moments during the day when the teacher and children work together to choose the theme, activities and materials Excelente 5 The teacher forsters and helps the children to choose the theme, activities and materials to be used throughout the day Observations:

16. EVOLUTION OF THE SCALE The Quality Scale is now in its fifth version, each version incorporating adjustments based on what was learned from applying the scale and as a result of a continuous dialogue with the intersectorial working group. In its first version the scale had 40 indicadors (and corresponding items); in the third, 45 and in the present version 54. Along the say, three additional dimensiones were added: accompaniment, health, and children with special needs. The main increase in the number of indicators was in the dimension of the educational process giving it more weight in the scale.

17. Results from the application of the Quality Scale (Version 3) in an evaluation of the Mexican Schools of Quality Program

18. The Schools of Quality Program (PEC) Objectives: improve school quality, autonomy and management, with parental/community involvement Modest funding to individual schools based on: DiagnosisStrategic Plan---Proposal Participation voluntary Funding from federal government, matched by states Annual review, possible funding up to 5 years Participation at level of about 5% of preschools in 2004- 2005

19. Characteristics of the study Evaluation and accompaniment Two samples: –S 1 4 centers in each of 26 states Visits at the beginning and end of the 2003-2004, school year and at the end of the 2004-2005 andy 2005-2006 school years (tres años) –S2 2 additional centers per state and two additional states. Visits atthe beginning and end of 2004-2005, and at the end of 2005-2006 (two years)

20. Metodología 3-person teams: PEC, Preschool education and the Pedagogical University 2-day visits: observation and interviews based on the ECCP plus field diaries, fotos (and in some cases videos) Analysis of results at the national level and for each center Accompaniment with feedback (3 sessions) aimed at improving plans and actions

21. Some Results

22. PEC contributed to improving the quality of preschools participating in the study

23. PEC has a greater influence on quality at the level of the center than in classrooms Begin 2003-4 End 2005-6 Difference Total 3.27 3.91 +.64 Center 3.34 4.15 +.81 Resources 3.27 4.16 +.89 Ed. Process 3.35 4.18 +.83 Ed. Management 3.41 4.23 +.82 Family/community 3.34 3.85 +.51 3º Year classrooom 3.12 3.80 +.68 Resources 3.20 3.70 +.50 Ed. Process 3.19 3.86 +.67 Ed. Management 2.97 3.58 +.61

24. STRENGTHS Improved infrastructure and materials Change management style – more colegial Improved attention to health and hygiene Improved statements of mission and vision preparations that support the educational process (mission and vision, diagnoses and planning)

25. AREAS REQUIRING IMPROVEMENT Identification and incorporation of the interests of the children; opportunities for children to propose ideas and act upon them. Activities that promote active learning: exploration, manipulation and reflection Attention to cultural diversity Resolution of conflicts The school´s relationship with parents and the community Supervision and accompaniment

26. After three years, the gap in quality between centers that entered with high and low quality has started to narrow

27. REFLECTIONS When defining quality it is important to think beyond the availability of schools and resources and beyond narrowly defined child outcomes. The most important dimension of quality is the educational process focussed on the interaction of teachers and students, mainly in the classroom. This process should be consistent not only with recognized principles of learning but should also with anagreed-upon (negotiated) vision of the society in which we want to live.

28. Reflections (2) Successfully changing the educational process from one that is mainly instructive to one that is constructive is a long-term project. A key element in obtaining that change (and others) is a process of human and technical accompaniment that goes beyond inspection or supervision, treating directors and teachers as subjects rather than objects.

29. REFLECTIONS (3) The Quality Scale helps the educational community reflect on and reexamine policies and programs at national, state and school levels by providing diverse and systematic information on a variety of dimensions.

30. REFLECTIONS (4) A major challenge for the Mexican educational system is to close the quality gap between higher and lower quality centers. This challenge is greatest in rural areas, often indigenous, where the conditons that support quality education are not in place.