Presentation on theme: "Examining Early Child Development in Low-Income Countries: A Toolkit for the Assessment of Children in the First Five Years of Life Lia Fernald, Ph.D."— Presentation transcript:
1 Examining Early Child Development in Low-Income Countries: A Toolkit for the Assessment of Children in the First Five Years of LifeLia Fernald, Ph.D.Patricia Kariger, Ph.DPatrice Engle, Ph.DAbbie Raikes, Ph.D.Not sure how you want to add in the authors or critical input.…
2 Acknowledgements Inspiration & funding from the World Bank Barbara Bruns, Sophie Naudeau, Harold Alderman, Ariel FitzbeinExternal reviewersFrances Aboud, McGill UniversitySantiago Cueto, Catholic University, PeruEd Frongillo, University of South CarolinaJane Kvalsvig, University of Kwa-Zulu Natal, South AfricaAnn Weber, University of California, BerkeleyPaul Wassenich, University of California, BerkeleyMichelle Neuman, The World BankMary Eming Young, The World BankCollaboratorsEmanuela Galasso, The World BankLisy Ratsifandrihamanana, MadagascarLourdes Schnaas, Mexican Institute of PerinatologyResearch assistantsRobin Dean (UC Berkeley), Kallista Bley (UC Berkeley), Melissa Hidrobo (UC Berkeley), Anna Moore (Cal Poly)Photo credits for photographs included in presentationLia Fernald, Emanuela Galasso, Lisy Ratsifandrihamanana, Ann Weber, Tricia Kariger
3 Today Importance of measuring child development Domains of development to be measuredTheoretical decisions in selecting instrumentsModification, adaptation and standardization of existing testsCreation of new testsTraining and quality controlConclusions and recommendations
4 Today Importance of measuring child development Domains of development to be measuredTheoretical decisions in selecting instrumentsModification, adaptation and standardization of existing testsCreation of new testsTraining and quality controlConclusions and recommendations
5 Introduction: Why measure child development? >200 million disadvantaged children worldwideI don’t have to tell anyone here that over 200 million children under the age of 5 worldwide are not fulfilling their potential for growth, cognition or socio-emotional development.The question of how to intervene for these children is of paramount importance, and is a primary focus of my research. I am also interested in examining how inequalities in socio-economic status contribute to nutritional outcomes in children and adults, and how nutritional conditions then relate to critical functional outcomes, such as cognitive and motor development in children, and physical activity and health outcomes in adults.I’ll start today by providing a theoretical overview of my research, and then I will delve into describing results from a specific intervention in Mexico.Percentage of disadvantaged children under 5 years old by country in 2004Grantham-McGregor et al., Lancet (2007)
6 Ecological model of child development Adapted representation of Bronfenbrenner’s ecological model of child development (Wortham, 2007)
7 Conceptual framework Environmental factors Direct&IndirectEffectsFrom Walker and al. Lancet, 2007Environmental factorsPsychosocial risks: harsh disciplinary techniques, maternal depressionBiological risks: malnutrition and infectious diseasesPoverty and socio-cultural factors increase likelihood of both types of risks
8 Timeline of development Timing of human brain development, from Grantham-McGregor, et al., 2007Early childhood is characterized by developmental spurts and plateausSkills emerge at different rates and ages
9 Differential risk and vulnerability Children’s development from 0-5 is dependent on quality of early environments and relationship with caregiver.Young children growing up in poverty are disproportionately exposed to a wide range of risk factors:Poor nutritionLess stimulating learning environmentsPoor sanitationStressful life eventsExposure to environmental risks
10 Poverty and cumulative risk Number of risk factors increases over time.Cumulative effect of risk factors becomes more evident as children get olderHigher cumulative levels of risk are associated with:Poorer cognitive developmentPsychological distress and behavioral problemsSlower and lower quality communicative development
11 Cultural norms and development Cultures have a wide range of values for when and how skills and abilities develop in children.As school becomes more universal, however, the necessary skills become more consistent across cultures.Through modification and adaptation, every effort must be made to ensure that tests are fair for all children assessed.
12 Today Importance of measuring child development Domains of development to be measuredTheoretical decisions in selecting instrumentsModification, adaptation and standardization of existing testsCreation of new testsTraining and quality controlConclusions and recommendations
13 Domains of development CognitiveLanguageMotorExecutive function/self-regulatorySocial/emotionalDomains are overlapping and mutually influencingEvery effort should be made to include multiple domains when assessing children’s development
14 Cognitive skillsCognitive skills include: Analytical skills, mental problem-solving, memory, and early mathematical abilitiesIndicators:Children near school age: knowledge of letters and numbers, ability to retain information in short term memory, knowledge of key personal informationChildren in school: knowledge of letters and numbers, reasoning, problem-solving, memory, and mathematical abilities
15 Executive functionDefined as fluid abilities or processes that are engaged when a person is confronted with a novel situation, problem or stimulusBoth cognitive and emotional processes are involvedCognitive: remembering arbitrary rules and other non-emotional aspects of the taskEmotional: inhibition or delayed gratificationIndicators:Working memoryInhibition of behavior or responses as demanded by the task (e.g. not opening a box until a bell rings)Sustaining attention as required or ability to switch attention as necessary (e.g. Shifting focus from the color of a test stimulus to the shape of the stimulus)“NIGHT”“DAY”
16 Language developmentEarly indicators (infancy): babbling, pointing, and gesturing. Use maternal report during this period.Later indicators (preschool years): production and understanding of words, ability to tell stories, identify letters, comfort and familiarity with books. Can use direct assessment.Quality and speed of development highly dependent on quality of caregiving environment
17 Motor skillsLarge motor: acquisition of movements that promote an individual’s mobility (useful to measure in young children)Contributing factors: brain and neuromuscular maturation, physical growth, caregiving practices, opportunities to practice emerging skillsFine motor: involves hand eye coordination and muscle control (e.g. drawing, holding utensils, etc.) (more relevant for older children)
18 Socio-emotional development First two years: relationships with caregivers, attachment, trust, and early strategies for dealing with negative feelingsPreschool years: social competence, behavior management, social perception, self-regulatory abilities
19 Today Importance of measuring child development Domains of development to be measuredTheoretical decisions in selecting instrumentsModification, adaptation and standardization of existing testsCreation of new testsTraining and quality controlConclusions and recommendations
21 Key Questions in Selecting Instruments What are the goals of the assessment/evaluation?What dimensions of child’s development do you expect to be affected by the intervention?What developmental systems are most vulnerable at a given age range?What are immediate outcomes and longer term outcomes?What are the mechanisms at work?What physiologic processes are influenced by iodine/iron/poverty?What are key elements of context that must be considered in selecting the test?Urban/rural, level of poverty, parent education.At what level will effect be measured?Individual? Household? Population (then consider test such as EDI)?How will the sample be selected?Population sample? Sub-sample?What is the analytic plan?Are norms relevant and/or available? Will a cut-off score be used?
23 Screening versus ability test Screening tests: brief assessments to identify children who are at risk of having development problemsInexpensive, quick, and relatively easy to administerClassify children into categoriesCutoffs used in one population to classify children should not be applied to another population!Ability tests: longer tests that assess the maximum skill level for a child at any given ageContinuous scores that can be used to compare children’s developmental levels with more precision
25 Types of assessments: Direct tests Pros:Data are gathered first handData can be less biased than parental reportsPotentially wider range of outcomes can be assessedMany of the “cons” can be overcome with careful planning and preparationCons:Young children can be difficult to test (sleeping, hungry)Testers need a lot of training and oversightAccuracy depends on testing demands and child must be familiar with parameters (e.g. best v. worst)
26 Types of assessments: Parent report ProsEasy to administer and require minimal training and instructionOften are quick and easy to complete and to scoreParents can become involved and express concernsOften correlate well with direct assessmentsTeachers can be an additional source of information as children get olderConsParents and teachers may artificially inflate scoresParents may not accurately report abilitiesParents and teachers may have different interpretations of items in different cultures
27 Types of assessments: Observation Types of observation: Naturalistic observation, Sampled observation, Structured situationPros:Highly validMeasures behavior in an identified contextCan provide additional or confirmatory information for other types of assessmentsCons:Requires a lot of time and trainingNeed to identify if culturally appropriateDifficult coding since observational codes and definitions are not always clearly defined
29 Other constraints to consider Budget: Tests can be very expensive (e.g. $1000 for Bayley); administration time is a budget issue, too.Copyright issues: Must obtain permission for most tests.Time allocated for testing: Direct assessment v. parent rep.Training: Capacity for administration.Test setting: Set-up, lighting, noise, observersCapacity of respondent: Education/knowledge of parentLanguage and cultural differences: Words used in testing materials, approach used for testing (e.g. speedy response)Materials: Must be familiar and/or available (e.g. mirror, ball)
30 Ethical risks and responsibilities All assessment protocols must be reviewed and approved by an ethical review boardAccuracy and validity are extremely important especially if test scores are being used to identify children “with delays”Follow-up (e.g. referrals for at-risk children) should be mandatory even in the context of a developing country.
31 Today Importance of measuring child development Domains of development to be measuredTheoretical decisions in selecting instrumentsModification, adaptation and standardization of existing testsCreation of new testsTraining and quality controlConclusions and recommendations
32 Instruments: Modifying and adapting No test is “culture free”Construct bias (e.g. test doesn’t measure “intelligence” the same way in both cultures)Method bias (e.g. procedures are unfamiliar and differentially affect responses)Item bias (e.g. individual test items do not translate well)Existing tests that are reliable and valid can be used across different cultures but they must be modified and adapted to achieve:Linguistic equivalenceFunctional equivalenceCultural equivalenceMetric equivalence (level of difficulty)
33 Preparatory work for test adaptation Involve local professionals to gather information relating to linguistic, cultural and technical details that could be relevant.Psychologists, community health workersEarly childhood educatorsProduce an accurate translationTranslation and back-translationReview, comparison, correctionPilot translated version to explore possible areas of confusion
34 Steps for test adaptation Adapt test content to local contextMake as many changes as necessary while maintaining the intended “meaning” of the itemExamples
35 Example: Modifying Peabody test Change from dollarsto AriaryRemove stairs frombannister
37 Example: Modifying Stanford Binet Most materials could be used as intendedDescription of picture included automatic washing machine – changed to traditional wash boardCognitive development was assessed for children older than 36 months using three sub-scales (long-term memory, short-term, or working memory, and visual integration) from the Spanish language version of the Revised Woodcock-Muñoz test. The scales have been used to evaluate effects of early childhood nutritional interventions and early health insults on cognitive development in children, and were selected because they have shown sensitivity to an income intervention in low income families, and to outcomes in children born low birth weight.WHAT EACH TEST IS
38 Example: Modifying Leiter test Replace ram with pigReplace car with tractorReplace flagReplace straight hair with curlyReplace reindeer with ox
39 Example: Modifying ASQ When in front of a large mirror, does your baby smile or coo at herself?
40 Example: Modifying ASQ When in front of a large mirror, does your baby reach out to pat the mirror?
42 Steps for test adaptation, cont’d Adapt administration proceduresTester (e.g. affect, responsivity, sensitivity, development of rapport, willingness to change environment)Test environment (e.g. materials, table, chair, lighting, sound, observers, other distractions)Test procedures (e.g. accuracy of parent response, clarity of instructions)
43 Example: Modifying ASQ Will caregivers make accurate assessments of their children’s development?We added 5 demonstration items toProvide children a chance to demonstrate behaviors that may not be easily observed (looking at pictures in a book; looking in a mirror)Act as a validity check of parent responses
44 Example: Modifying ASQ Without showing him first, does your child point to the correct picture when you say, “Show me the kitty” or ask, “Where is the dog?”GIVE THE PICTURE TO THE CAREGIVER AND ASK HER TO SHOW IT TO HER CHILD.SAY TO THE CAREGIVER: “I know children do not always do what they are asked, but let’s see if he will do this for us today. Go ahead and ask [CHILD] to show the kitty, dog, ball or shoes.” INSTRUCT THE CAREGIVER NOT TO POINT TO ANY PICTURES. YOU CAN ALLOW ABOUT ONE MINUTE FOR THE CHILD TO DEMONSTRATE THE BEHAVIOR.
45 Example: Modifying ASQ Can we adapt the majority of items across all cultures?We added clarifications where items seemed ambiguousDoes your baby get into a crawling position by getting up on her hands and knees? [BABY DOES NOT HAVE TO CRAWL, BUT MUST BE ABLE TO MAINTAIN SELF ON HANDS AND KNEES.]Does your child drink without help from a cup or glass, putting it down again with little spilling? [CHILD CAN DRINK ALONE FROM A CUP WITHOUT SPILLING TOO MUCH.]
46 Today Importance of measuring child development Domains of development to be measuredTheoretical decisions in selecting instrumentsModification, adaptation and standardization of existing testsCreation of new testsTraining and quality controlConclusions and recommendations
47 Requirements for creating a new test Involve an inter-disciplinary research teamUse a representative sample for testing items and test cohesionConduct a detailed analysis of the instrument’s psychometric propertiesDevelop norms or standards that represent typical development in the population under study
48 Examples of new tests Africa Asia Latin America Kilifi Developmental Inventory -assesses psychomotor development in a resource-limited settingGrover-Counter Scale of Cognitive Development -developed in South Africa to assess the level of cognitive functioning of children 3-10 years with impaired verbal skillsAsiaCambodian Development Assessment Test – measures level of cognitive, social, motor, and academic development based on country specific standardsLatin AmericaTest de Desarollo Psicomotora -developed in Chile, it evaluates child development in motor function, coordination, and languageEscala de Evaluacion del Desarrollo Psicomotor – screening measure of language, social, coordination, and gross motor skills. Norms and cutoffs developed for Chile.
49 Using the “Standards” approach How to develop a set of StandardsDefine domainsWithin each domain, define a set of standards or goalsFor each standard, outline the specific objectives and indicators for each age levelPros of Standards approachCulturally appropriateProcess can be informativeCons of Standards approachTime-intensive and requires long term follow-upIndicators are not necessarily translated into a testNeeds to be done slowly and carefullyExample from Vietnam for children 5-6 years oldNOTE: UNICEF has worked with over 40 countriesto develop Standards
50 Today Importance of measuring child development Domains of development to be measuredTheoretical decisions in selecting instrumentsModification, adaptation and standardization of existing testsCreation of new testsTraining and quality controlConclusions and recommendations
51 Steps for training Establish “gold standard” interviewer Involve local psychologistsEstablish “gold standard” interviewerTest for inter-rater reliabilityTest for rater accuracy
52 Reliability and accuracy Inter-rater reliabilityRater accuracy
53 Today Importance of measuring child development Domains of development to be measuredTheoretical decisions in selecting instrumentsModification, adaptation and standardization of existing testsCreation of new testsTraining and quality controlConclusions and recommendations
55 Broad recommendations Assess characteristics of the child that the intervention is intending to affect.Make sure to measure variables that could also be contributing to the outcomes (e.g. maternal responsiveness, home environment)Decide on the type of outcome measure that is appropriate for the evaluation.Rely upon multiple measures of children’s development.Include assessments of executive function and socio-emotional developmentConsider the cultural context and how it may affect children’s development and school readinessAlways work with local collaborators!Look for national level tests where possible and use parent/teacher report when possible.Begin following children early in life.
56 Criteria for being recommended Psychometrically adequate, valid and reliable;Balanced in terms of number of items at the lower end to avoid children with low scores;Enjoyable for children to take (e.g. interactive, colorful materials);Relatively easy to adapt to various cultures;Easy to use in low-resource settings, e.g. not requiring much material;Not too difficult to obtain or too expensive;Able to be used in a wide age range.
57 Specific recommendations: 0-36 mo. Continuous measure, direct assessmentBayley Scales of Infant DevelopmentNationally adapted test (e.g. Indian version of Bayley II)Kilifi Executive Function TasksContinuous measure, maternal reportMacArthur Communicative Development InventoriesNationally adapted test (e.g. Turkish Guide for Monitoring Child Development)Screening test, direct assessmentDenver Developmental Screening testNationally developed test (e.g. EEDP from Chile)Screening test, maternal reportAges and Stages Questionnaire (ASQ)
58 Specific recommendations: 3-5 y. Cognitive developmentStanford BinetBritish Ability Scales II Early YearsWechsler Preschool and Primary Scales of Intelligence (WPPSI)
59 Specific recommendations: 3-5 y. Language developmentPeabody Picture Vocabulary Test (PPVT) or spanish version: Test de Vocabulario de Imagines PeabodyReynell Developmental Language Scale
60 Specific recommendations: 3-5 y. Executive functionLeiter Examiner ScaleDay/Night Task and Backward Digit TaskBRIEF-P (Parent/teacher report)Social and behavioral developmentStrengths and DifficultiesAchenbach Child Behavior Checklist
61 Contact info and further readingCONTACT INFORMATION:Lia Fernald:Patricia Kariger:Patrice Engle:FURTHER READING:Peña, E. D. (2007). Lost in translation: Methodological considerations in cross-cultural research. Child Development, 78(4),Snow, C.E. and Van Hemel, S.B. (Eds) Early Child Assessment: Why, What, and How. Washington D.C.: The National Academies Press. 2008Young, M.E. and Richardson, L.M. (Eds) Early Child Development: From Measurement to Action. Washington D.C.: The World Bank