Presentation is loading. Please wait.

Presentation is loading. Please wait.

Early Learning Standards, Measurement and Regulation Parallel Session C: August 28, 2012.

Similar presentations


Presentation on theme: "Early Learning Standards, Measurement and Regulation Parallel Session C: August 28, 2012."— Presentation transcript:

1 Early Learning Standards, Measurement and Regulation Parallel Session C: August 28, 2012

2 Speakers Prof Nirmala Rao, Professor, Faculty of Education, The University of Hong Kong – Using Early Learning and Development Standards in the East Asia Pacific Region Dr. Moni Day, Visiting Associate Professor CECED, AUD – Early Learning Standards: Indian Case Dr. Maxine Bernstein, Professor Emeritus, School of Education, Tata Institute of Social Sciences, Hyderabad – Emergent to Early Literacy

3 Introductory Remarks Why is the assessment of early development and learning important? What are Early Learning and Development Standards (ELDS)?

4 Why is the assessment of early development and learning important? Monitor child development in the context of poor school readiness and learning outcomes. Track the development of vulnerable and at- risk children in parts of Asia, sub-Saharan Africa 4

5 What are Early Learning and Development Standards? Statements of expectation for “what children should know and be able to do” Five Broad Domains of School Readiness have been widely accepted: Physical Health, Well-Being, and Motor Development Social & Emotional Development Approaches Toward Learning Language, Literacy & Communication Cognition and General Knowledge

6 Standards Language: Terms & Definitions There are four levels: – Level 1: Domain Level 1.A. Sub-Domain – Level 2: Standard – Level 3: Indicator – Level 4: Preparatory Learning Activities

7 Example of an Early Learning Standard for 4-yr child Indicator: Child is able to follow directions that involve a two or three step sequence of actions Domain: Language, Literacy & Communication Sub-domain: Communication Standard: Children should be able to listen to and understand language Preparatory Learning Activity: Give oral directions and play a game like “caregiver says” Make the children give simple directions to each other (Kagan & Britto, 2006)

8 Example of an Early Learning Standard for 4-yr child Domain: Physical Health & Well-being & Motor Development Sub-Domain: Health & Personal Care Standard: Children should be able to practice basic care routine Indicator: Child gets drink of water without assistance Preparatory Learning Activity: Offer plenty of opportunities to the child to take care of self (Kagan & Britto, 2006)

9 Using Early Learning and Development Standards in the East Asia-Pacific Region Nirmala Rao Faculty of Education The University of Hong Kong South Asia Regional Conference on Early Childhood Care and Education Policies and Practices: Towards 2015 and beyond. August 27 – 29, 2012, New Delhi

10 10 Overview 1.ELDS in the East Asia-Pacific Region 2.ARNEC project: Method & Findings 3.Conclusions

11 11 ELDS in the East Asia-Pacific Region 7 countries in the East Asia-Pacific Region had developed and/or validated ELDS for their countries with the support of UNICEF 11

12 12 ARNEC Project ARNEC commissioned HKU team to do project on ELDS: Nirmala Rao, Pat Engle, Sun Jin, Marie Ng, Yvonne Becher, Diana Lee, Carrie Lau and Zhang Li Develop draft Early Child Development Scales based on the Early Learning Development Standards (ELDS) of countries of the East Asia and Pacific Region The Scales should have dimensions of child development which all countries consider valuable

13

14 14 ARNEC Project Phase I Select items to be used in the scale (desk review) Phase II Pilot test in small samples in as many countries as possible. Revise and redo test based on results Phase III Representative sample in as many countries as possible. Final revision and creation of age-based norms

15 15 Phase I We set out to develop the scales for the specific purposes (not a screening test) Created a data file with all of the indicators, domains, standards, country and age Decided on common categories and sub-categories across countries based on analysis of data file Established reliability on definitions of categories Selected most common indicators for creating items for pilot testing

16 Established an Indicator Database 1738 indicators for children aged 3, 4, and 5 from seven countries’ ELDSs Developed categories + sub-categories 7 categories Determined # of indicators in each category Based on the proportion of the total # of indicators in each category Preparation of the database Converted indicators to items Form a 100-item measure Selected indicators in each category To construct a 100-item regionally-sensitive measure  Focus on rationale and process

17 17 Preparation of the database Stage 4 – Selecting indicators Established and applied decision rules and criteria for including or excluding indicators We decided to select 100 indicators from the list of 1710 indicators. Why 100 items ?

18 Categories # of sub- categories Total # of indicators Approaches to learning 1297 Cognitive Development Cultural Participation and Knowledge Language and Emergent Literacy Motor Development Health, Hygiene and Safety Socio-Emotional Development Total

19 Categories # of sub- categories Total # of indicators % of total indicators in each category Approaches to learning Cognitive Development Cultural Participation and Knowledge Language and Emergent Literacy Motor Development Health, Hygiene and Safety Socio-Emotional Development Total

20 20

21 1. Approaches to Learning 1. Approaches to Learning EAP-ECDS Structure 2. Cognitive Development 2. Cognitive Development 22 (27.66% of total indicators) 3. Cultural Knowledge and Participation 3. Cultural Knowledge and Participation 10 (12.57% of total indicators) 4. Language and Emergent Literacy 4. Language and Emergent Literacy 15 (16.61% of total indicators) 5. Motor Development 5. Motor Development 9 (8.54% of total indicators) (8 separate; 1 incorporated in another item) 9 (8.54% of total indicators) (8 separate; 1 incorporated in another item) 6. Health, Hygiene and Safety 6. Health, Hygiene and Safety 7. Socio-Emotional Development 7. Socio-Emotional Development 9 (8.89% of total indicators) 16 (20.06 % of total indicators) 9 (5.67% of total indicators ) (8 separate; 1 incorporated in another item) 9 (5.67% of total indicators ) (8 separate; 1 incorporated in another item) Total 90 21

22 22 Stage 5 – Converting indicators to items CodeSub-category Indicator selected CountriesAges LA16 Draws/writes without an example Draws a human figure (head, eyes, mouth, trunk, arms, legs, etc.) without prompts Philippines(4); Cambodia(5); Thailand(4;5); 4, 5

23 23 Materials1.A piece of paper; A pencil ProcedureAssessor asks child to draw a complete picture of him / herself. InstructionHere is a pencil and paper. I would like you to draw a complete picture of yourself standing. Scoring Criteria and Comments Head and/or trunk plus one other feature. Recognizable arms and/or legs. Recognizable hands and/or feet. One recognizable facial feature (eyes, mouth, nose, or ears). At least one additional recognizable facial feature (eyes, mouth, nose, or ears) □ 0 □ 1 □ 0 □ 1 □ 0 □ 1 □ 0 □ 1 □ 0 □ 1

24 24 S tage 5 – Converting indicators to items CodeSub-category Indicators Selected CountriesAge SS01 Shows or uses respectful behavior Tells how to pay respect to elders (Ex. clapping hands, bowing, etc.). Uses polite words with adults (by answering thank you, hello, good bye etc.). Cambodia, Laos, Mongolia, Philippines, Thailand, Vanatu, Vietnam 3;4;5

25 25 MaterialsTwo pictures of: 1.A sitting child and a standing adult in bus (can be changed by country). 2.An adult giving a nicely wrapped gift to a child (only two persons in picture) ProcedureAssessor shows one picture at a time and asks child what the child in the picture should say. Instruction1.(for Picture 1) Look at this picture. Look at this child. Here is a grandmother. What would you say or do if you were the child? 2.(for Picture 2) In this picture, the adult is giving a very nice gift to this child. What would you say to the adult if you were this child? Scoring Criteria and Comments Correctly answered Question 1. Correctly answered Question 2. □ 0 □ 1 □ 0 □ 1

26 26 Revised feedback on draft categories, subcategories, indicators and items at January workshop HKU team developed items, scoring criteria and instructional manual. How we did this? Iterative process Panel of experts reviewed drafts materials and appropriateness of methods and suggested changes

27 27 Summary (Phase I) Bottom-up process of defining instrument from existing concepts – based on existing ELDS from EAP countries Created a data file with all of the indicators, domains, standards, country and age Decided on common categories and sub-categories across countries based on analysis of data file Obtained reliability on definitions of categories Selected most common indicators for creating items for pilot testing

28 28 Phase II: Pilot studies 1.To field test items in three diverse countries 3. To develop a final version of the Scales which can be used in many countries in Phase III 2.To modify the Scales based on the results and feedback

29 Pilot study Mongolia Fiji China Sent draft to countries Field trials Translation & material checking Adapted EAP-ECDS for each country Modification Field Consultancy Support

30 30 Specific Suggestions Use testing materials which are familiar to the child Use language which child can understand Follow the standard instructions and procedures Give children different ways to respond : actions not always words Establishing and maintaining rapport

31 31 Field training workshops

32 32 Enhance reliability and validity Structure equivalence Inter-assessor reliability Representative sampling Test administration – In preschool or in home depending on context – Tester records notes regarding items and administration – Parental questionnaire

33 33 Schedule for pilot studies Countries received the test package May Test administration in Mongolia and Fiji Jun. – Aug. Jun. Consultations and support to Mongolia and Fiji Oct. & Forward Oct. & Forward 1.Data collection and field support in China 2.Data analyses of Mongolia and Fiji data and report writing Aug. -Sep. 1.Preparation of data collection in China 2.Data Entry in Mongolia and Fiji

34 34 Samples in three sites UrbanRuralTotal (in school) BoysGirlsBoysGirlsBoysGirls 3 years (13) 19(16) 4 years (20) 22(17) 5 years (15)18(17) Fiji 3 years (7)20(9) 4 years (5)14(9) 5 years (20)33(15) China 3 years10 20(20) 4 years10 20(20) 5 years10 20(20) (Total: 120)

35 35 Duration of assessment Mongolia Fiji China (urban) 3 year olds 4 year olds 5 year olds m m m 30m 45m m m m (two sessions)

36 36 Questions for Phase II validation Can this scale measure achievement of standards/competence/ability? – Are the items appropriate? Discriminative Difficulty level How do the countries differ? – Scoring on the scales – Difference across age groups

37 37 Strategy Step 1 Model Selection Compare the fit by different models Step 2 Item Analysis Item discrimination Item variance (difficulty) Step 3 Country comparisons Responses pattern Scores

38 38 Model Selection Can this scale measure ability/attainment of ELDS? Item Response Theory (IRT) – Assume a latent ability variable – Latent variable measured by observed variables – Observed variables are dichotomous or polytomous

39 39 Models considered Rasch Model Two-parameter logistic latent trait model Birnbaum’s three-parameter model

40 40 Model selected AICLRTP-Value Rasch Logistic (2 parameter) <0.001 Birnbaum’s

41 41 Item Difficulty & Discrimination: Cognitive Development Top 5 Easiest Items CD12_2Sat on the chair CD22_4Correctly put the toy under the chair CD12_1Brought the block (or other small item) to assessor CD3_1Correctly put one block inside the box CD22_3Correctly put the toy on the chair Top 5 Hardest Items CD22_2Correctly put the toy on the left side of the chair CD5_2Correctly picks 7 blocks on the paper CD17_3Correctly arranged all the pictures in right order CD5_3Correctly picks 15 blocks on the paper CD5_4Correctly picks 29 blocks on the paper

42 42 Item Difficulty & Discrimination: Cognitive Development

43 43 Total Information: Cognitive Development

44 44 Item Difficulty & Discrimination: Socio-Emotional Development Top 3 Easiest Items SED1_1Knows full formal name SED80_1Consistently followed the rules during assessment SED25_2Correctly point to the HAPPY face Top 3 Hardest Items SED24_4 Correctly answered “what help” in the losing mummy situation SED26_2Correctly gave an answer to help stop crying SED24_5Correctly gave a second answer for “what help” in the losing mummy situation

45 45 Item Difficulty & Discrimination: Motor Development Top 3 Easiest Items MD36_1Poured water and sipped from an open cup independently MD35_1Walk forward: fell off or side-stepped three times or less MD41_1Successfully strung large beads Top 3 Hardest Items MD38_4Hit the target on third attempt MD38_2Hit the target on first attempt MD38_3Hit the target on second attempt

46 46 Item Difficulty & Discrimination: Language and Emergent Literacy Top 5 Easiest Items LEL45_4Correctly identified playing (with ball) LEL43_5Correctly identified cleaning teeth LEL45_2Correctly identified reading LEL43_1Correctly identified combing hair LEL45_1Correctly identified running Top 5 Hardest Items LEL56_2Copied triangle LEL56_3Copied rectangle LEL48_3 Mentioned another one or more events or happening of one event in logical order LEL46_3Linked all four pictures with some cohesive evidence LEL54_3Drew recognizable hands and/or feet

47 47 Item Difficulty & Discrimination: Health, Hygiene and Safety Top 3 Easiest Items HHS58_1Could pretend to clean his/her face and hand HHS58_2Could pretend to brush his/her teeth HHS60_2Could take the smock off Top 3 Hardest Items HHS59_1Mentioned washing hands after using the toilet HHS66_2Correctly identified confectionary as unhealthy food HHS66_1Correctly identified chips as the unhealthy food

48 48 Item Difficulty & Discrimination: Cultural Knowledge and Participation Top 3 Easiest Items CKP74_1 Correctly clapped one line according to the rhythm of the song CKP74_2Correctly clapped one more line CKP74_3Correctly clapped another one ore more lines and refrain Top 3 Hardest Items CKP70_1Mentioned informing an adult CKP70_2Mentioned returning the toy to the owner CKP71_3Correctly named another festival

49 49 Item Difficulty & Discrimination: Approaches to Learning Top 3 Easiest Items ATL84_1Showed positive emotion ATL83_1Beginning: stays on task ATL84_4Spontaneous positive interaction with assessor Top 3 Hardest Items ATL83_6End: not want to stop tasks ATL83_5End: not easily distracted ATL83_3Beginning: not want to stop tasks

50 50 Country Comparisons: Cognitive Development

51 51 Discussion Psychometric issues – Limited data – Factor analysis difficult and unreliable – Time consuming and challenging to develop culturally appropriate items

52 52 Items localized Changed the task: Item 44 Phoneme awareness task  syllable detection task (Shu, Hong, & McBride-Chang, 2008) Changed the pictures: Item 64 Vegetable  Fruit Changed the pictures: Item 65 French fries  Twisted fried bread

53 53 Item 17: The pictures are not clear enough. The logic in the story is not specific. Item 19: The pictures are not clear enough. The word “weather” is a bit abstract for children. The instructions should be simplified. Items to be revised

54 54 Items to be revised Item 22: The instructions are ambiguous.

55 55 Difficulties in Assessment Children felt tired during the process of the assessment.  Let children go out of the test room if they feel tired. The assessors’ conversations with the children beyond the instructions prolonged the whole testing.  Avoid too many conversations and always keep children focused on the test. The assessors unintentionally revealed the recording sheet to children which made children nervous. How to motivate children during assessment?  Give children enough oral encouragement, but not small gifts

56 56 Conclusions Test based on culturally sensitive ELDS; first effort in the region Test is reasonably valid and reliable and can meet need for a tool to monitor child development at the population level Valuable effort to achieve both cultural appropriateness and item equivalence in cross-cultural measurement 56

57 57 THANK YOU! 57


Download ppt "Early Learning Standards, Measurement and Regulation Parallel Session C: August 28, 2012."

Similar presentations


Ads by Google