1. Faculty of Education The University of Hong Kong Early Childhood Development in the East Asia Pacific: Development & Findings from the East Asia Pacific – Early Child Development Scales (EAP-ECDS) Sun Jin Assistant Professor, Faculty of Education Hong Kong Institute of Education Yvonne Becher Head of Psychological Services, The Child Development Centre, Hong Kong Presentation at the ARNEC –Conference 2014, Manila December 4, 2014 1

2. Faculty of Education The University of Hong Kong EAP-ECDS A collaborative effort by -UNICEF -ARNEC -Open Society Foundation -Faculty of Education, University of Hong Kong 2

3. Faculty of Education The University of Hong Kong

4. Faculty of Education The University of Hong Kong Overview 1.Rationale for developing the East Asia Pacific – Early Child Development Scales (EAP-ECDS) 2.Process of Development – Phases 1, 2 and 3 3.Overall Findings 4.Lessons Learned 5.Potential Uses 6.Conclusions 4

5. Faculty of Education The University of Hong Kong Rationale for Developing the EAP-ECDS 1.Need to invest in young children – UNICEF’S Early Learning and Development Standards (ELDS) Standards are statements of expectation for “what children should know and be able to do” Five Broad Domains of School Readiness have been widely accepted: Physical Health, Well-Being, and Motor Development Social & Emotional Development Approaches Toward Learning Language, Literacy & Communication Cognition and General Knowledge 5

6. Faculty of Education The University of Hong Kong Rationale for Developing the EAP-ECDS 2.Implementation of Standards 3.Emphasis on evidence-based decision making What are the differences between children who attend and those who do not attend early childhood programmes? How important is quality? Which programmes should be scaled up? To answer these questions, we need a tool to measure early childhood development but… 6

7. Faculty of Education The University of Hong Kong Rationale for Developing the EAP-ECDS There are no globally accepted tests for ECD Culturally and contextually appropriate assessment of ECD is important to:  monitor child development in the context of poor school readiness and learning outcomes  track the development of vulnerable and at-risk children  analyse the impact of early childhood policies and programmes on children 7

8. Faculty of Education The University of Hong Kong Phase I To select items to be used in a tool to assess early child development in the EAP region – These items should be based on the Early Learning and Development Standards (ELDS) of countries in the region 8

9. Faculty of Education The University of Hong Kong 9

10. Faculty of Education The University of Hong Kong Established an Indicator Database 1738 indicators for children aged 3, 4, and 5 from seven countries’ ELDS Developed categories + sub-categories 7 categories Determined # of indicators in each category Based on the proportion of the total # of indicators in each category Development of the Items Converted indicators to items Form a 100-item measure Selected indicators in each category To construct a 100-item regionally- sensitive measure  Focus on rationale and process

11. Faculty of Education The University of Hong Kong Converting indicators to items CodeSub-category Indicator selected CountriesAges LA16 Draws/writes without an example Draws a human figure (head, eyes, mouth, trunk, arms, legs, etc.) without prompts Philippines(4); Cambodia(5); Thailand(4;5); 4, 5

12. Faculty of Education The University of Hong Kong Materials1.A piece of paper; A pencil ProcedureAssessor asks child to draw a complete picture of him / herself. InstructionHere is a pencil and paper. I would like you to draw a complete picture of yourself standing. Scoring Criteria and Comments Head and/or trunk plus one other feature. Recognizable arms and/or legs. Recognizable hands and/or feet. One recognizable facial feature (eyes, mouth, nose, or ears). At least one additional recognizable facial feature (eyes, mouth, nose, or ears) □ 0 □ 1 □ 0 □ 1 □ 0 □ 1 □ 0 □ 1 □ 0 □ 1

13. Faculty of Education The University of Hong Kong Converting indicators to items CodeSub-category Indicators Selected CountriesAge SS01 Shows or uses respectful behavior Tells how to pay respect to elders (Ex. clapping hands, bowing, etc.). Uses polite words with adults (by answering thank you, hello, good bye etc.). Cambodia, Laos, Mongolia, Philippines, Thailand, Vanuatu, Vietnam 3;4;5

14. Faculty of Education The University of Hong Kong MaterialsTwo pictures of: 1.A sitting child and a standing adult in bus (can be changed by country). 2.An adult giving a nicely wrapped gift to a child (only two persons in picture) ProcedureAssessor shows one picture at a time and asks child what the child in the picture should say. Instruction1.(for Picture 1) Look at this picture. Look at this child. Here is a grandmother. What would you say or do if you were the child? 2.(for Picture 2) In this picture, the adult is giving a very nice gift to this child. What would you say to the adult if you were this child? Scoring Criteria and Comments Correctly answered Question 1. Correctly answered Question 2. □ 0 □ 1 □ 0 □ 1

15. Faculty of Education The University of Hong Kong Revised feedback on draft categories, subcategories, indicators and items HKU team developed items, scoring criteria and instructional manual. How we did this? Iterative process Panel of experts reviewed drafts materials and appropriateness of methods and suggested changes

16. Faculty of Education The University of Hong Kong Phase II: Pilot studies 1.To field-test items in three countries 2.To modify the Scales based on the results and feedback 3.To develop a revised version of the Scales which can be used in many countries in Phase III 16

17. Faculty of Education The University of Hong Kong Pilot study Mongolia Fiji China Send draft to countries Field trials Translation & checking Modification Field Consultancy Support

18. Faculty of Education The University of Hong Kong Questions for Phase II Evaluate the effectiveness of the Scales – Reliability Cronbach’s alpha: >.70 – Validity Content validity: expert review Age validity: older children scoring significantly higher than younger children – Appropriateness Cross-cultural differences Item discrimination Item facility 18

19. Faculty of Education The University of Hong Kong Item Difficulty & Discrimination: Cognitive Development 19 Top 5 Easiest Items CD12_2Sat on the chair CD22_4Correctly put the toy under the chair CD12_1Brought the block (or other small item) to the assessor CD3_1Correctly put one block inside the box CD22_3Correctly put the toy on the chair Top 5 Hardest Items CD22_2Correctly put the toy on the left side of the chair CD5_2Correctly picked 7 blocks on the paper CD17_3Correctly arranged all the pictures in the right order CD5_3Correctly picked 15 blocks on the paper CD5_4Correctly picked 29 blocks on the paper

20. Faculty of Education The University of Hong Kong The Revised Scale after Phase II 20 2. Cognitive Development 21 items 4. Language and Emergent Literacy 16 items 6. Health, Hygiene and Safety 9 items 7. Socio-Emotional Development 15 items Total 85 items 5. Motor Development 7 items 1. Approaches to Learning 10 items 3. Cultural Knowledge and Participation

21. Faculty of Education The University of Hong Kong Objective of Phase III Validate the EAP-ECDS in 6 countries in the region 21

22. Faculty of Education The University of Hong Kong Sampling From June to December 2013, EAP-ECDS assessments were carried out in six countries Samples included both boys and girls residing in urban and rural settings, attending preschool programmes and those who did not; the number of children assessed in each country ranged from 900 to 1803 Sampling in all countries were stratified by child’s Age, Gender and Urbanicity 22

23. Faculty of Education The University of Hong Kong Country adaptations Translated versions of the EAP-ECDS Instruction Manual, Scoring Form and Parent Questionnaire 23 CountryTranslated versions in the country’s native language(s) CambodiaKhmer ChinaChinese MongoliaMongolian Papua New GuineaTok Pisin Timor-LesteTetum VanuatuBislama

24. Faculty of Education The University of Hong Kong Country adaptations: Item 15 Arrange picture according to temporal order 24 Cambodia PNG China

25. Faculty of Education The University of Hong Kong Country adaptations: Item 17 Predicts season or weather 25 Cambodia Vanuatu PNG

26. Faculty of Education The University of Hong Kong Country adaptations: Item 52 Letters, characters / symbols 26 Cambodia China 大，小，人，山，水，日，月，木，树，手，牛，羊。 Mongolia А, Н, О, М, Б, У, С, Э, Т, И, Р, Л;

27. Faculty of Education The University of Hong Kong Country adaptations: Item 54 Wordless picture book 27 Cambodia China Mongolia Timor-Leste

28. Faculty of Education The University of Hong Kong Country adaptations: Item 61 Child in bathroom with a sink 28 Timor-Leste Mongolia Cambodia China PNG Vanuatu

29. Faculty of Education The University of Hong Kong Country adaptations: Item 63 Signs 29 Cambodia China Vanuatu

30. Faculty of Education The University of Hong Kong Country adaptations: Item 72 Picture of a teddy bear sitting outside the front door of a local-type store 30 Vanuatu PNG China

31. Faculty of Education The University of Hong Kong Country adaptations: Item 77 A picture with national flags 31 Cambodia Mongolia China PNG Timor-Leste Vanuatu

32. Faculty of Education The University of Hong Kong Country adaptations: Item 78 An adult throwing large amounts of garbage into a clean river 32 China Vanuatu PNG

33. Faculty of Education The University of Hong Kong Parent Questionnaire Demographic Information about the family – Family asset – Family members – Family learning support The child’s early learning and development The child’s health and habit 33

34. Faculty of Education The University of Hong Kong Overall Findings 34

35. Faculty of Education The University of Hong Kong Important points These findings are based on six countries in our region that vary widely in human development indicators, size, early childhood policy, challenges and opportunities. Countries varied in sampling procedures, and samples varied in terms of preschool attendance. We did not make cross-country comparisons like in the PISA, PIRLS, TIMMS – we think it is inappropriate to do so. 35

36. Faculty of Education The University of Hong Kong Demographic and Human Development Indicators in the six countries 36

37. Faculty of Education The University of Hong Kong Sample for EAP-ECDS validation by country, age and gender 37

38. Faculty of Education The University of Hong Kong Number of children with and without Early Childhood Education (ECE) across countries 38

39. Faculty of Education The University of Hong Kong Scale Validation Cronbach’s alpha Content validity Comparisons across different domains and groups – 3 (Age) × 2 (Gender) × 2 (Urbanicity) MANOVA Consistency between children’s performance and parents’ assessment Item analysis – Appropriateness: discrimination and facility – Differential item function analysis: no systematic bias – Item information curves and test information function curves for each domain 39

40. Faculty of Education The University of Hong Kong Internal consistency of the EAP-ECDS domain scores across countries 40

41. Faculty of Education The University of Hong Kong Items for parents’ rating of children’s competence 41

42. Faculty of Education The University of Hong Kong Correlations between parent ratings and EAP-ECDS domain and total scores 42

43. Faculty of Education The University of Hong Kong Predicting early childhood development in the East Asia Pacific 43 Coefficients and 95% confidence intervals P-values Intercept-7.40 (-13.69, -1.11) 0.02 Pre-school attendance 6.52 (4.10, 8.76) <0.001 Household Asset0.81 (0.53, 1.09) <0.001 Mother’s education0.67 (0.50, 0.87) <0.001 Sex (Girl as reference) (-1.54, -0.46) <0.001 Age12.28 (11.95, 12.62) <0.001

44. Faculty of Education The University of Hong Kong Age, gender, urban-rural differences in total EAP-ECDS score across countries 5Y > 4Y > 3Y Gender imbalance Urban-rural imbalance 44

45. Faculty of Education The University of Hong Kong Age differences in total EAP-ECDS score across countries 45

46. Faculty of Education The University of Hong Kong Gender differences in total EAP-ECDS score across countries 46

47. Faculty of Education The University of Hong Kong Urban-rural differences across countries in total EAP-ECDS score 47

48. Faculty of Education The University of Hong Kong Regression coefficients from multi-level model for global and country specific effects of preschool attendance on the EAP-ECDS 48

49. Faculty of Education The University of Hong Kong Country-specific effects of the impact of attendance in an early childhood programme on early child development and learning 49

50. Faculty of Education The University of Hong Kong Participation in ECCE Urban-rural disparities in participation Older children and those of better-educated parents were more likely to be enrolled in an early learning programme than other children. In almost all countries, the children who attended ECCE went to kindergarten (In Timor-Leste, about half went to community/drop-in centres). 50

51. Faculty of Education The University of Hong Kong Home Learning Environments 30 to 60 per cent of parents reported engaging in early learning-related activities with children at home. Educated parents tended to support early learning at home more than other parents. Mothers were more involved than other family members with the exception of Timor-Leste 51

52. Faculty of Education The University of Hong Kong Children’s Health and Habits Almost all children were vaccinated Age was best predictor of health facilitation practices (taking children to the clinic) Older children and girls tended to have better health and hygiene habits than other children 52

53. Faculty of Education The University of Hong Kong Reported Health Problems Urban parents and better educated mothers were more likely to report that their children had health problems. It is not clear whether – urban children suffer poorer health – urban parents are more aware of children health issues, and are more likely to report health concerns, or – facilities are not available in rural areas 53

54. Faculty of Education The University of Hong Kong Potential uses to use evidence to -decrease urban/rural gaps; -decrease gender imbalances; -promote parental involvement; -advocate early childhood education and care; -guide ECCE curriculum development/ review; -… 54

55. Faculty of Education The University of Hong Kong Lessons learned 55 need a low-resource tool (hard to access areas!) never skip language selection & (back-) translation process (meanings get lost) be culturally appropriate (item flexibility vs. standardisation) consider direct vs. indirect assessment train tool users/ administrators (training? how?)

56. Faculty of Education The University of Hong Kong Conclusions First regional tool Bottom-up approach to development of assessment tool Enormous resources put into ensuring context- sensitivity of items, and developing a psychometrically robust tool There are many challenges associated with ensuring cross-cultural equivalence of items 56

57. Faculty of Education The University of Hong Kong Thank you for your attention!