Background of the Early Communication Indicator for Infants and Toddlers Dale Walker, Judith Carta, and Charles Greenwood

This Work n Early Childhood Research Institute on Measuring Growth and Development n Other partners: –University of Minnesota –University of Oregon n Developing indicators toward a set of nationally validated child outcomes from birth to age 8 years to produce a comprehensive performance measurement system

University of Kansas ECI Development Project Staff for Children birth to 3 years : Judith Carta, Charles Greenwood & Dale Walker With: Jane Atwater, Gayle Luze, Deborah Linebarger, Carol Leitschuh, Ken Parsley, Annessa Staab and Gabe Cline

Objectives of Training: n Understand the background of the ECI and this type of assessment n Learn how to administer and score the ECI and how to become a certified administrator n Understand how to use ECI data for monitoring individual progress and evaluating programs n Learn about the website and how to enter data n Learn how to train others

Current Focus on Outcomes- Based Program Evaluation More emphasis needed on child outcomes More emphasis needed on child outcomes Programs (and individual staff members) need to know when they are making a difference in moving children toward outcomes Programs (and individual staff members) need to know when they are making a difference in moving children toward outcomes

Most available approaches to program evaluation: Not linked to individual child progress on outcomes Not linked to individual child progress on outcomes Child data not used in making program decisions Child data not used in making program decisions –Not frequent enough –Not accessible enough

Indicators of child progress are needed to determine if children are growing in their development

What’s an Indicator? What is an Individual Growth and Development Indicator (IGDI)?

What’s an indicator? n Way of monitoring progress toward outcomes: knowing whether you are on-track n Tells you when a change is needed n If you make changes, helps you “see” the results

Helpful features of the thermometer indicator: n Quick measure n Gives instant information n Tells a lot about an important general outcome: Health Status n Can repeat it frequently n Measures intervention results

Pediatrician’s Standard Growth Charts: Height and Weight Indicators

Features of growth charts Easy to do Easy to do Relatively inexpensive Relatively inexpensive Repeatable Repeatable Can indicate potential problem and effectiveness of an intervention Can indicate potential problem and effectiveness of an intervention

General Outcome Measurement Not a new concept in education Not a new concept in education –Recognized in special education practice and research –Reading Rate (CBM) –DIBELS (Dynamic Indicators of Basic Early Literacy Skills)

Outcomes and Indicators to Assess Progress Outcomes are general, socially desired abilities/capabilities Outcomes are general, socially desired abilities/capabilities Indicators are sensitive markers of progress Indicators are sensitive markers of progress –Way of seeing if children (and programs) are moving toward outcomes. –Share features of other effective indicators: –Quick, easily repeatable, provide quick feedback –Show change easily

Expressive Communication General Outcome One of 15 general child outcomes validated by a national sample of parents (n=351) and professionals (n=672) One of 15 general child outcomes validated by a national sample of parents (n=351) and professionals (n=672) Expressive communication was most highly rated outcome Expressive communication was most highly rated outcome Others include, social, movement, problem- solving and adaptive behaviors Others include, social, movement, problem- solving and adaptive behaviors

Expressive Meaning Outcome “The child uses gestures, sounds, words, or sentences to convey wants and needs or to express meaning to others.”

General ECI Administration: How it’s Done Administration Procedures Administration Procedures Administration Procedures Administration Procedures –Toy-play setting with familiar adult –6 minute assessment sessions –Adult’s role is to facilitate play and follow child’s lead –Set-up/clean-up/put away Alternate Toy Forms Alternate Toy Forms Observational Recording Procedures frequency of communicative behavior (key skill elements) tallied to get rate of total communication Observational Recording Procedures frequency of communicative behavior (key skill elements) tallied to get rate of total communicationRecording ProceduresRecording Procedures ©2003 Juniper Gardens Children’s Project Toy Form A: House Toy Form B: Barn

Vocalizations Multiple Words Single WordsTotal CommunicationGestures Key Skill Elements What does the ECI Measure?

Recording Example

How we know it works: Using the ECI for individual and program monitoring How we know it works How we know it works Does it grow? Does it grow? Is it reliable? Is it reliable? Is it valid? Is it valid? Is it predictive? Is it predictive? Individual progress monitoring Individual progress monitoring Program evaluation Program evaluation ©2003 Juniper Gardens Children’s Project

How Did Key Skill Elements Grow over 9 Months for Children in Their 1st Year? ©2003 Juniper Gardens Children’s Project

How Did Key Skill Elements Grow for Children in their 2 nd Year of Life? ©2003 Juniper Gardens Children’s Project

How Did Key Skill Elements Grow For Children in Their 3 rd Year of Life? ©2003 Juniper Gardens Children’s Project

Did Total Communication Grow and Differ by Age Cohort? ©2003 Juniper Gardens Children’s Project

Did we detect differences for children with disabilities? ©2003 Juniper Gardens Children’s Project

Does this approach work? Is it sensitive to growth? Is it sensitive to growth? Scores show change over time as children age and develop 0 – 12 mos: M = 4.8; 13 – 24 mos. M = 8.3; 25 – 36 mos. M = 11.7 Is it reliable? Is it reliable? Trained observers agree on what communication they record. (90% overall) Is it valid? Is it valid? ECI indicators correlate with standardized and parent report measures. (PLS; CCM) Is it predictive? Is it predictive? ECI indicators correlate with preschool literacy and communication indicators. ©2003 Juniper Gardens Children’s Project

How the ECI is used for Progress Monitoring: Levels of Growth Assessment Bench Marking (quarterly – every 3 months) Bench Marking (quarterly – every 3 months) –All children assessed –Who is making or not making expected progress? –Who needs additional support? –How is the program doing? Strategic Monitoring (monthly) Strategic Monitoring (monthly) –Is more support accelerating growth? –Is a change in intervention supports needed? Intensive Care Monitoring (weekly) Intensive Care Monitoring (weekly) –Is growth accelerating? –Is a change in intervention supports needed? ©2003 Juniper Gardens Children’s Project

IGDIs ESAs IGDI Decision Making Model Monitor Identify/ Validate Need For Intervention Generate Intervention Strategies Implement Intervention Evaluate Intervention Effectiveness Monitor Families Outcomes ©2003 Juniper Gardens Children’s Project

Framing Intervention Solutions: Ruling out, Ruling in History with child? History with child? Medical issues? Medical issues? Family/caregiver concerns? Family/caregiver concerns? What child knows and does? What child knows and does? Settings, activities, routines, curriculum? Settings, activities, routines, curriculum? Challenges to implementing intervention? Challenges to implementing intervention? ©2003 Juniper Gardens Children’s Project

Identification

' ' ' ' ' ' ' ' ' '' ' ' ' ' ' ' ' '' ' ' Weeks Rate Per Minutes Communications per MinuteExpected Growth ' The ECI Tells Us When to Intervene and Whether or Not Intervention Is Working or Program is Effective BaselineInterventionModified Intervention ©2003 Juniper Gardens Children’s Project

How Approach is Used to Inform Decision-Making at Individual Levels Home visitors use individual child data to monitor child progress Home visitors use individual child data to monitor child progress Use child data to evaluate effectiveness of interventions and program Use child data to evaluate effectiveness of interventions and program Use child data to plan with parent(s) Use child data to plan with parent(s) ©2003 Juniper Gardens Children’s Project

Effects of Individual Interventions ©2003 Juniper Gardens Children’s Project

How Approach is Used to Inform Decision-Making at Program and State Levels ECI data available to State Head Start/Early Head Start Administrators, intervention staff and families ECI data available to State Head Start/Early Head Start Administrators, intervention staff and families Program administrative staff, Early Head Start Coordinator and SRS use ECI for : Program administrative staff, Early Head Start Coordinator and SRS use ECI for : –Program Evaluation –Program Planning –Quality Improvement –Inform Public and Funding Sources about EHS Progress ©2003 Juniper Gardens Children’s Project

Monitoring Programs Program data aggregated up from individual data Program data aggregated up from individual data Assess all or some children in a program Assess all or some children in a program Measure every 3 months: (6, 9, 12, etc.) Measure every 3 months: (6, 9, 12, etc.) Each child has a growth trajectory (rate of growth and mean level) Each child has a growth trajectory (rate of growth and mean level) Calculate the average (or program) trajectory (rate of growth and mean level) Calculate the average (or program) trajectory (rate of growth and mean level) Children below expected rate of growth - assess monthly and provide (ramp up) intervention. Children below expected rate of growth - assess monthly and provide (ramp up) intervention. Continue monthly progress monitoring and continue intervention decision making Continue monthly progress monitoring and continue intervention decision making ©2003 Juniper Gardens Children’s Project

P1 = Growing at.31 responses per month P2 = Growing at.51 responses per month P3 = Growing at.60 responses per month ©2003 Juniper Gardens Children’s Project

Program Progress Overall Child Rate of Growth Overall Child Rate of Growth Overall Child Rate of Growth Overall Child Rate of Growth –Ave. Slope of Communications for individual children and groups of children in programs –Mean Communications at different periods of development for individuals and groups of children Child Growth Rate Child Growth Rate –By Gender –By IFSP Status By IFSP StatusBy IFSP Status –By Language Status By Language StatusBy Language Status Staff Operations Staff Operations –Staff Qualifications Summary Staff Qualifications SummaryStaff Qualifications Summary –Interobserver Agreement Summary Interobserver Agreement SummaryInterobserver Agreement Summary

Better assessment can lead to improvements in intervention Can give practitioners a focus for their efforts Can give practitioners a focus for their efforts Can help interventionists “see” when they’re making a difference Can help interventionists “see” when they’re making a difference Can help practitioners know more quickly when a change is necessary Can help practitioners know more quickly when a change is necessary Can help directors understand when programs need improvements Can help directors understand when programs need improvements

Sources Greenwood, C. R., Luze, G. J., & Carta, J. J. (2002). Assessment of intervention results with infants and toddlers IV. In A. Thomas, & J. Grimes (Eds.), Best Practices in School Psychology (Vol. 2, pp ). Washington DC: National Association of School Psychology. Greenwood, C. R., Luze, G. J., & Carta, J. J. (2002). Assessment of intervention results with infants and toddlers IV. In A. Thomas, & J. Grimes (Eds.), Best Practices in School Psychology (Vol. 2, pp ). Washington DC: National Association of School Psychology. Greenwood, C. R., Luze, G. J., Cline, G., Kuntz, S., & Leitschuh, C. (2002). Developing a general outcome measure of growth in movement for infants and toddlers. Topics in Early Childhood Special Education. Greenwood, C. R., Luze, G. J., Cline, G., Kuntz, S., & Leitschuh, C. (2002). Developing a general outcome measure of growth in movement for infants and toddlers. Topics in Early Childhood Special Education. Luze, G. J., Linebarger, D. L., Greenwood, C. R., Carta, J. J., Walker, D., Leitschuh, C., & Atwater, J. B. (2001). Developing a general outcome measure of growth in expressive communication of infants and toddlers. School Psychology Review, 30(3), Luze, G. J., Linebarger, D. L., Greenwood, C. R., Carta, J. J., Walker, D., Leitschuh, C., & Atwater, J. B. (2001). Developing a general outcome measure of growth in expressive communication of infants and toddlers. School Psychology Review, 30(3), McConnell, S. R. (2000). Assessment in early intervention and early childhood special education: Building on the past to project into the future. Topics in Early Childhood Special Education, 20, McConnell, S. R. (2000). Assessment in early intervention and early childhood special education: Building on the past to project into the future. Topics in Early Childhood Special Education, 20, McConnell, S. R., Priest, J. S., Davis, S. D., & McEvoy, M. A. (2002). Best practices in measuring growth and development for preschool children. In A. Thomas, & J. Grimes (Eds.), Best practices in school psychology IV. (Vol. 2, pp ). Washington DC: National Association of School Psychology. McConnell, S. R., Priest, J. S., Davis, S. D., & McEvoy, M. A. (2002). Best practices in measuring growth and development for preschool children. In A. Thomas, & J. Grimes (Eds.), Best practices in school psychology IV. (Vol. 2, pp ). Washington DC: National Association of School Psychology. Priest, J. S., McConnell, S. R., Walker, D., Carta, J. J., Kaminski, R. A., McEvoy, M. A., Good, R. H., III, Greenwood, C. R., & Shinn, M. R. (2001). General growth outcomes for children: Developing a foundation for continuous progress measurement. Journal of Early Intervention, 24 (3), Priest, J. S., McConnell, S. R., Walker, D., Carta, J. J., Kaminski, R. A., McEvoy, M. A., Good, R. H., III, Greenwood, C. R., & Shinn, M. R. (2001). General growth outcomes for children: Developing a foundation for continuous progress measurement. Journal of Early Intervention, 24 (3), Individual Growth and Development Indicators (IGDIs): Assessment that guides intervention for young children. In M. Ostrosky & E. Horn (Eds.). Assessment: Gathering meaningful information. The Young Exceptional Children Monograph Series #4. Longmont, CO: Sopris West. Carta, J., Greenwood, C. R., Walker, D., Kaminski, R., Good, R., McConnell, S. R., & McEvoy, M. (2002). Individual Growth and Development Indicators (IGDIs): Assessment that guides intervention for young children. In M. Ostrosky & E. Horn (Eds.). Assessment: Gathering meaningful information. The Young Exceptional Children Monograph Series #4. Longmont, CO: Sopris West.