Community Input Discussions: Measuring the Progress of Young Children in Massachusetts November 2009

2 Statement of Intent Massachusetts is in the early stages of developing a statewide system to measure developmental progress of its young children EEC is engaging parents, providers, program administrators, teachers, higher education institutions, and policy makers to build a responsive approach This initiative is separate from (and would not replace) developmental information that programs gather about children to use for curriculum planning and to individualize instruction

3 College Visits to Date June 23 rd : Bristol Community College July 1: Mount Wachusett Community College July 8 th : Holyoke Community College August 5 th : Berkshire Community College August 12 th : Bunker Hill Community College August 13 th : Mass Bay Community College August 27 th : Northern Essex Community College September 10 th : Quinsigamond Community College September 22 nd : North Shore Community College September 24 th : Greenfield Community College November 9 th : Middlesex Community College

4 Questions for Stakeholder Input What are your hopes for measuring the developmental progress of young children in state? How do you hope information will be used? What are the most important things to measure about school readiness? What are some of the challenges you foresee in moving forward with this effort?

5 Measuring School Readiness Across the Country Massachusetts is joining rapidly growing trend to understand school readiness 29 states currently collect statewide data on children’s progress These efforts are often directly connected to state-funded preschool efforts

6 Why Are Statewide Data Needed? To inform policy makers about the benefits of and impact of investments in early childhood education and care in Massachusetts To better understand school readiness gaps(s) for subgroups of children To inform statewide policy development Data will not be used for high stakes “testing” of young children or providers

7 Statewide Measures of School Readiness vs. Child Assessments and Screenings Currently, many providers are already using a developmental assessment or screening tool to inform practice and individualize instruction Providers are currently using a variety of assessment measures UPK grantees are required to use one of four assessment systems Creative Curriculum Developmental Continuum Ages & Stages High Scope Child Observation Record (COR) Work Sampling System Statewide measurement of school readiness is not intended to be used to replace program-level assessment practices

8 Statewide Measures of School Readiness vs. Child Assessments and Screenings Different purposes Statewide system: information about the success of all children in Massachusetts Program-level assessments: information for parents and caregivers about individual children Different level of information about child Statewide system: measure a small number of indicators of school readiness Program-level assessments: comprehensive look at child progress across all developmental domains

9 Statewide Measures of School Readiness vs. Child Assessments and Screenings Different usefulness to providers Statewide system: for broader policy purposes Program-level assessments: to help provider support each child’s growth and development Other differences Statewide system: children will be anonymous when data are reported

10 Design Options for Statewide Measure: Key Issues to Consider WHAT to measure about child development WHO to measure WHEN to measure HOW will measures/information on children be collected WHAT ELSE to measure about home environments, program characteristics, and community context

11 Design Options for Statewide Measure: Key Issues to Consider WHAT: how broadly should we measure children’s development – which domains are most important? WHO: all children and programs, or just a sample? WHEN: longitudinal data in preschool only (ages 3 and 4) or linking with school data (K and/or grade 1)? HOW: providers trained to administer measures, outside evaluators, and/or other informants?

12 Other State Approaches All states look at development either at one point in time or over time, usually using different samples of children at each age All states interested in measuring school readiness skills, although the skills that they measure vary Types of measures used vary (standardized assessment vs. developmental observations, number of development areas measured) Where system is ongoing, states involve providers in data collection

13 Other States: Approaches Using Providers as Data Collectors MD: Ongoing assessment of all children at school entry by kindergarten teachers, using Work Sampling System PA: Ongoing measures of all children three time per year during preschool and kindergarten by providers, using Work Sampling System NC: Snapshot of school readiness information using variety of measures from a sample of principals, kindergarten teachers, parents, and children just entering kindergarten, coordinated by research firm

14 Key Issues Raised To Date (based on six stakeholder meetings) Engage parent and provider support for initiative to ensure maximum participation Make sure purpose of initiative is clear Ensure information collected about children does not reflect negatively on families Consider how to support providers and minimize burden if they will be on the front lines of the measurement effort Consider how to provide context for data on development of vulnerable subgroups of children Consider linking child assessment data to program quality data to better understand linkages between quality and child outcomes

15 Key Decisions Moving Forward: WHAT to measure? Narrow vs. broad measurement Tension between desire to measure “whole child” and what is feasible to collect Which skills/outcomes to measure Focus on outcomes that research tells us are related to success in school such as: Academic skills in reading, writing, and/or math Social skills Cognitive and behavioral self-regulation

16 Key Decisions Moving Forward: WHO to measure? Could measure as broad a population as possible or target more narrowly Might oversample particular subgroups of interest Effort will likely be limited to preschool children in licensed and license-exempt settings Probably not feasible to access children who are in informal and/or unlicensed care settings or who are not in out-of-home care at all

17 Key Decisions Moving Forward: HOW to measure? Providers as assessors Advantages: cost, may help with getting parent permission, providers learn about their children’s skills Disadvantages: concerns about bias if providers assess their own children, need to train large number of providers, difficult for providers to find time to conduct quiet standardized assessment Outside assessors Advantages: can be trained to reliability, no public concerns about partiality, possibility of building on early childhood education infrastructure in state to develop group of assessors Disadvantages: cost of conducting assessments, cost of training Parents as assessors Advantages: builds parent buy-in, not expensive Disadvantages: public perception of bias, some concepts might be hard to explain to parents, may be difficult to get parents to return this information

18 Key Decisions Moving Forward: Consider Integration/Overlap with Other Assessment Efforts UPK grantees using one of four specified child assessment tools Many other programs are using either one of these four tools or another measure School district programs using Battelle in spring of 3-year-old year with children who will be evaluated for special needs Other efforts?

19 Key Decision Points Moving Forward: WHEN to measure? Options for data collection schedule before school entry Measure children once, at the end of preschool, to describe school readiness of preschoolers Measure children twice, at the beginning and end of preschool, to describe school readiness but also to see whether children who start out below average are closing the gap over time Measure children three times, once during the 3-year-old year and fall and spring of the 4-year-old year. This design provides more information on progress over time Looking at same group of children over time vs. different groups of children each time Although following the same children would allow us to look at individual growth over time, this is a very expensive undertaking

20 Feedback to Date Hopes for Assessment Hopes that assessment will: ●Increase our ability to help children with severe behavioral needs. ●Be long term and follow children over time to see progress of children statewide. ●Will be conducted in the child care setting, which is a natural environment for them. ●Offer a credit bearing system for training educators on assessment and mentoring through partnerships with community colleges.

21 ●Critical thinking and confidence. ●Social/Emotional competence. ●Math and Early Literacy. ●Cultural Identity. ●Consider use of work sampling, as it provides a rich experience. ●Consider tracking last two places child lived for context. Feedback to Date Measurement

22 Feedback to Date Challenges and Concerns ●Tasking already overburdened educators with conducting assessments ●Will this interfere with focusing on play? Will educators “teach to the test”? ●Children learn at their own rate. How can we standardize this? ●How can we use this to address the summer learning loss? ●Standardized Assessments may not give a complete picture of the child.

23 Feedback to Date Challenges and Concerns (cont.) ●How can we use information we have already gathered (such as UPK)? ● Will we provide training for educators to learn the skills to be objective assessors? ● What happens after the initial assessment? Do we continue to track children? ● Will Infants and Toddlers be part of the assessment? ● Need to correct for the fact that the child population may be unstable in a program. ● EEC should consider paying for professional development days twice a year for FCC providers who accept vouchers to offer training on assessment.

24 Feedback to Date Challenges and Concerns (cont.) There should be a statewide campaign to help parents understand: ●The importance and value of assessment. ●Early brain development and early literacy. ●Children are “evaluated” from birth (APGAR) and this is simply another process for collecting important information. ●Appropriate expectations regarding extreme academic demands at an early age.

25 Questions for Stakeholder Input What are your hopes for measuring the developmental progress of young children in state? How do you hope information will be used? What are the most important things to measure about school readiness? What are some of the challenges you foresee in moving forward with this effort?