1. Department of Early Education and Care Initiatives September, 2011
Understanding the Learning Needs of Young Children: Formative Assessment for Quality and Child Outcomes
Department of Early Education and Care Initiatives
September, 2011

2. Overview of Presentation
Presentation will highlight the Department’s initial strategies relative to the Race to the Top – Early Learning Challenge and Comprehensive Assessment
Focus is on the Department’s work in the area of formative assessment
Panelists will respond with the perspective from the field

3. Closing the Achievement Gap: Accountability for Quality and Outcomes
Teacher Quality
Program Quality (QRIS)
Community and Family Context
Child Outcomes (formative and summative assessment)

4. Race to the Top: Early Learning Challenge
The Race to the Top – Early Learning Challenge application covers five key areas that “represent the foundation of an effective early learning and development reform agenda that is focused on school readiness and ongoing academic success.”
Successful State Systems;
High-Quality, Accountable Programs;
Promoting Early Learning and Development Outcomes for Children;
A Great Early Childhood Education Workforce; and
Measuring Outcomes and Progress
All of these reports are available on EEC’s website under Research, Planning, and Publications

5. Measuring Outcomes and Progress (ELC)
ELC Key Definitions
States are encouraged to implement comprehensive data systems and to use the data to improve:
instruction,
practices,
services, and
policies
States will be rewarded for implementing kindergarten entry assessments statewide that:
provide information across all domains of early learning and development,
inform efforts to close the school readiness gap, and
inform instruction in the early elementary school grades.
All of these reports are available on EEC’s website under Research, Planning, and Publications

6. Comprehensive Assessment System (ELC)
A coordinated and comprehensive system of multiple assessments, each of which is valid and reliable for its specified purpose and for the population with which it will be used,
that organizes information about the process and context of young children’s learning and development in order to help Early Childhood Educators make informed instructional and programmatic decisions and
that conforms to the recommendations of the National Research Council reports on early childhood.
A Comprehensive Assessment System includes, at a minimum--
(a) Screening Measures;
(b) Formative Assessments;
(c) Measures of Environmental Quality; and
(d) Measures of the Quality of Adult-Child Interactions.
All of these reports are available on EEC’s website under Research, Planning, and Publications

7. Purpose of Assessment Systems
Child Assessment
Child assessment tools, which are becoming increasingly regarded as a marker of high quality programming, can be used as a valuable method for educators to improve their instruction to meet the needs of individual children as well as inform their general program practice.
Program Assessment
Program assessment tools are used to measure the quality of the program environment and teacher/child interactions. These tools are included in most QRIS systems.
All of these reports are available on EEC’s website under Research, Planning, and Publications

8. ELC Shared Definitions: Screening Measures
Screening Measures: Age and developmentally appropriate, valid, and reliable instruments that are used to identify children who may need follow-up services to address developmental, learning, or health needs in, at a minimum, the areas of:
physical health
behavioral health
oral health
child development
vision and hearing
Tool: ASQ and ASQ-SE
Purpose: Support parent understanding growth and development
Target Population: All children birth to 5
How Information is Used: To determine developmental delays
All of these reports are available on EEC’s website under Research, Planning, and Publications

9. Licensing Requirements
A written progress report must be prepared periodically on the progress of each child in the program
The program must offer parents a conference to discuss the content of the report. 
Educators must use progress reports to:
adapt the program to the children’s individual strengths, interests, and needs;
to maintain ongoing communication with the child’s family, and;
with parental permission, to facilitate the child’s transition to another program or to kindergarten, as appropriate.
Frequency:
For infants and children with identified special needs: every 3 months
For toddlers and preschoolers: every 6 months
For school age children: at least annually, at the midpoint of the child’s program year
Content: The progress report must be based on observations and documentation of the child’s progress in a range of activities over time
Children younger than school age: Cognitive, Social/Emotional, Language and Fine and Gross Motor and Life Skills 
School age children: child’s growth and development within the parameters of the program’s statement of purpose    
All of these reports are available on EEC’s website under Research, Planning, and Publications

10. ELC Shared Definitions: Assessment
Formative assessment: Assessment questions, tools, and processes that are:
(1) Specifically designed to monitor children’s progress in meeting the Early Learning and Development Standards;
(2) Valid and reliable for their intended purposes and their target populations;
(3) Linked directly to the curriculum
Tool: Work Sampling, High Scope, Creative Curriculum
Purpose: Guide and improve instructional practices in
Target Population: All Programs in QRIS
How Information is Used: To provide measures of growth in children and target individualized learning plans
All of these reports are available on EEC’s website under Research, Planning, and Publications

11. ELC Shared Definitions: Assessment
Norm referenced assessment: Assessment performed by educators, which are trained to reliability, to understand a child’s learning in comparison to that of their peers or sample population.
Tool: PPVT, EVT, Woodcock-Johnson, Social Emotional test
Purpose: To assess multiple domains
Target Population: Sample of children to be determined
How Information is Used: To support understanding of the well-being of children ages 0 -5
All of these reports are available on EEC’s website under Research, Planning, and Publications

12. Measures of Environmental Quality
Quality Rating and Improvement System Requirements (QRIS) - Measurement Tools
Programs in QRIS are required to use measurement tools at Level 2 (self-assessment), Level 3 and Level 4 (reliable raters)
Tool: Environmental Rating Scales (ITERS, ECERS, SACERS, FCCERS)
Purpose: Program Quality Measures
Target Population: All Programs in QRIS
How Information is Used: To support continuous improvement in the early education and care environments   
All of these reports are available on EEC’s website under Research, Planning, and Publications

13. Measures of Quality Adult-Child Interactions
Quality Rating and Improvement System Requirements (QRIS) - Measurement Tools
Programs in QRIS are required to use measurement tools at Level 2 (self-assessment), Level 3 and Level 4 (reliable raters)   
Tool: Arnett Caregiver Interaction Scale, and Classroom Assessment Scoring System (CLASS)
Purpose: Teacher and child interactions
Target Population: All Programs in QRIS
How Information is Used: To support continuous improvement and monitoring of high quality teacher and child interactions
All of these reports are available on EEC’s website under Research, Planning, and Publications

14. Overall EEC Screening and Assessment Strategy
Overall Strategy
Build a system of screening and assessments using the Early Learning Challenge Grant
Acknowledge that different age groups and/or capacity of the system will require different strategies 
 Screening Children Age Birth-5: Population Targets
At risk children
Infants and toddlers: ASQ and ASQ-SE will be implemented
Preschool children: In QRIS and Licensed programs; use CFCE for others (set tool undecided e.g. ASQ or ESI)
Kindergarten children: ESE and EEC will identify what screeners and referrals are currently in use and work with districts to capture the relevant information and track referrals.
Multiple points of screening to measure growth
Purpose
Understand capacity of communities to provide services to children identified. 
Provide opportunity for intervention prior to referrals to Early Intervention and Special Education
Talk with families about growth and development, how to interpret results
Assure access to resources/family education, if the screening leads to the identification of issues (catalogue of resources)
All of these reports are available on EEC’s website under Research, Planning, and Publications

15. Overall EEC Screening and Assessment Strategy
 Professionals targeted to administer the screening
Grantees who have a meaningful relationship with families (CFCE, Home Visiting, Public Schools, Maternal Health, Family Support grantees; can use ASQ and ASQ-SE)  
Embed ASQ in pediatricians’ offices through Reach Out and Read program
Screening Tools
The ASQ is promoted in the statewide in-home visiting and family engagement grant and the ASQ and ASQ-SE seem to be the tools of choice. 
ESE is currently surveying Full Day Kindergarten programs in the public schools regarding the tool they use for screening.
All of these reports are available on EEC’s website under Research, Planning, and Publications

16. Formative Assessment Strategy
Linked to QRIS quality definition
Ensure programs have access evidence based formative assessment for high need children, including training and linkage with higher education
Support statewide access to selected evidence based formative assessment tools; provide flexibility in measures used but support consistency 
Ensure information is shared with other programs including public schools when children transition; a student information system would enhance this
Use the state provision of the assessment tool as a link to freeing up resources in programs to support compensation

17. Norm Referenced Assessment Strategy
Use domains of literacy, numeracy and executive function as proxy’s for school readiness in MA.
Support for educators in administering tools to support professional development and compensation. 
Select a random sample of children in communities across the state through a combination of appropriate incentives and data infrastructure. 
To validate formative assessment
To determine effectiveness of community based strategies

18. Norm Referenced Summative Assessment
EEC Comprehensive Assessment Strategy
Formative Assessment
Norm Referenced Summative Assessment
Access to educational screenings and assessments for all children
Types: Work Sampling, High Scope, Creative Curriculum
Provider: Public schools, community programs
Program Environment (Adult/Child Interactions)
Types: PPVT, EVT, Woodcock Johnson Social/Emotional
Provider: Public schools, community programs
Types: Arnett, CLASS
Provider: EEC programs
Screening Assessments (e.g., ASQ, ASQ-SE)
450,000 children: in communities, EEC programs, and schools – including Child Find, CFCE programs

19. Early Education Screening and Assessment Activities

20. Preschool Screening and Assessment FY10/11
A report by Abt Associates found across program types programs scored high in emotional support and classroom organization, though performed significantly lower on instructional support.
Given the low results on instructional support, EEC sought to address this issue by supporting the implementation and utilization of research-based comprehensive child assessment systems and screening tools in preschool settings so that educators can individualize their instruction by child and improve program practice.
In fiscal years 2010 and 2011, EEC awarded Associated Early Care and Education an $800,000 Assessment Grant each year for the purpose of providing assessment and screening training and materials statewide.
The FY10 and FY11 Assessment Grant through Associated provided trainings on the following assessment and screening systems statewide:
Ages & Stages and Ages & Stages SE (screening),
Teaching Strategies Creative Curriculum,
Teaching Strategies GOLD,
High Scope COR, and
Work Sampling System.
All of these reports are available on EEC’s website under Research, Planning, and Publications

21. Preschool Screening and Assessment FY12
EEC will award grants to one or more Institutes of Higher Education (IHE) or partnerships to design and coordinate training on QRIS assessment and screening tools to support program improvement in settings serving children birth to 13 across the Commonwealth.
This grant will be coordinated with Educator and Provider Support networks to ensure locally that programs engaged in QRIS have access to and are being trained to integrate assessment, screening and use of the measurement tools into their programs and implementation of QRIS.
The grant will be used to develop modules for Environmental Rating Scales, Screening and Formative assessments tools that support the existing EPS grantees to offer similar Training using a training of Trainers (ToT) model to support sustainability and increased access for all providers
The grant will focus on the same assessment systems utilized in FY10 and FY11
All of these reports are available on EEC’s website under Research, Planning, and Publications

22. Assessment: Kindergarten Readiness
EEC and Elementary and Secondary Education (ESE) are collaborating to develop an aligned education system for children birth through age 8 including.
As one of the initial steps EEC is working to define measures of kindergarten readiness that can be used for at least two checkpoints in preschool and/or at the start of kindergarten.
Information from the assessments will illustrate children’s readiness level for kindergarten and be used to inform early education and care and kindergarten programs about trends in readiness and how to support individual children’s growth.
Additionally, data will be aggregated at the state level to make statements about all children’s preparedness for kindergarten across the Commonwealth.
This initiative will compliment work being done by the Departments to assign state assigned student identification numbers (SASIDS) to children statewide.

23. Kindergarten Readiness Assessment Model Design and Pilot FY11
In FY11 EEC hired New York University Child and Family Policy Center (CFPC) to conduct a Kindergarten Readiness Assessment Model Design and Pilot Project
CFPC is designing a model of formative and/or summative assessment that can be used in preschool and/or kindergarten in community and public school to demonstrate children’s kindergarten readiness level, by:
Providing child-level data to educators which can be used to inform classroom practice and individualize instruction for children;
Providing data that can be aggregated at the program level for site- based improvement and growth tracking;
Providing sample or complete data sets that can be aggregated at the state level to inform the school readiness level of children in MA.
CFPC trained preschool educators on administering the Peabody Picture Vocabulary Test, Woodcock Johnson, and social-emotional assessments with children under the supervision of the trainers.
Of 16 teachers from Head Start, day care and family day care,
14 attended two training sessions,
14 completed the prescribed practice with a range of 5-10 children, and
11 were “certified” as ready to continue to independently practice assessment within their local school context.

24. Kindergarten Readiness Assessment Model Design and Pilot FY12
Goal in FY12 is to certify 76 teachers over the next 12 months to:
Increase the system-wide capacity to assess children’s progress and readiness in ways that align with Early Learning Standard and well as Common Core Standards
Create a professional development program that trains teachers to administer selected measures, interpret the results, understand implications for practice and communicate the findings to parents as well as colleagues with whom they collaborate
Create a system for efficient entry, cleaning, and analysis of resulting data garnered from those assessments and interface with early childhood and care system data professionals to integrate this data handling into regular practice
Sites in Springfield, and possibly Holyoke, and Worcester.
All 76 teachers must complete 5-10 training assessments before becoming certified. Will use large sample of children who are 4-5 years old (N=380).
Certified teachers must conduct at least 10 assessments
Data on 760 children will be entered into a database system and analyzed for descriptive information in summer of 2012.
Proposals are due September 16th

25. Help Me Grow
The University of Connecticut Health Center has awarded a grant to EEC to replicate the Help Me Grow model in Massachusetts.
A key component of the Massachusetts Help Me Grow model is the use of the Ages and Stages Questionnaire (ASQ) screener to educate families about child development.
EEC purchased 10 Ages and Stages screeners (in English and Spanish) to be used in a pilot program with volunteer Coordinated Family and Community Engagement (CFCE) grantees.
Goal is to engage families to participate in the program, promote child development knowledge using the ASQ-3 and ASQ:SE tool, and link children, who are identified as needing further assessments, to community-based programs and services.
The 15 CFCE grantees selected for the pilot received training on the ASQ tool on August 3rd. Each pilot site received training on ASQ, a kit of materials and access to the on-line screening tool
All of these reports are available on EEC’s website under Research, Planning, and Publications

26. Early Education Screening and Assessment Activities: ESE Survey of Full Day Kindergarten Grantees (FY07)

27. Developmental Screening
Which of the following does the district use for screening children entering kindergarten? (total n=128)
Early Screening Inventory (ESI) 60% (n=77)
Other tool (*see below) 26% (33)
DIAL % (19)
BRIGANCE % (15)
Districts own Screening Tool 8% (10)
PPVT % (8)
DIAL-R % (7)
Ages and Stages Questionnaire 5 % (6)
DIBELS % (5)
PALS % (5)
DENVER II % (1)
* Other screening tools mentioned: PHELPS, DALLAS, Batelle, Daberon, Early Prevention of School Failure, Joilet, McCarthy, Hainsworth, SIB-R, and Fluherty

28. Curriculum
Which of the following do your programs use for curriculum? (n=128)
Dev. Appropriate Practice (NAEYC) 77% (98)
Other Curriculum (** see below.) 57% (73)
Everyday Math 22% (28)
TERC’s Math Investigations 20% (25)
Scott Forseman/ERI Reading 19% (24)
Harcourt Brace (Math and/or Reading) 16% (20)
Houghton Mifflin 14% (17)
Handwriting without Tears % (15)
Wilson’s FUNdations 11% (14)
Social-Emotional Curriculum % (13)
Science (e.g., FOSS, Scholastic) 9% (12)
Active Learning % (11)
District developed 46% (59)
Scott Forseman Math 8% (10)
Project Approach (Katz/Helms) 7% (9)
Reggio Emilia % (7)
High Scope % (5)
Scholastic EC Program 2% (3)
Creative Curriculum 2% (2)
Montessori Approach 2% (2)
OWL % (2)
Bank Street 1% (1)
Building Language for Literacy 1% (1)
Learningames % (1)
Marazon Systems % (1)
Step by Step 1% (1)

29. Assessment
If an assessment tool is used, which one do you use? (all that apply) (n=128)
Other Assessment*** 57% (n=76)
Own Assessment 53% (68)
DIBELS 48% (53)
DRA % (33)
Work Sampling System 27% (35)
Focused Portfolio 17% (22)
GRADE % (13)
None % (3)
Ages & Stages Questionnaire 2% (2)
High Scope/COR 1% (1)
Creative Curriculum’s assessment 1% (1)
***Assessment tools mentioned include: Standford Achievement Tests, Early Reading Inventory, Brigance, and many assessments that are provided with the formal curriculums noted in Question 2.
If you have developed your own tool, what kind of information did you start with? (n=128)
Based on our own knowledge of child development 38% (49)
Some other basis (i.e MA Curriculum Frameworks) 23% (29)
Based on a published tool % (25)

30. Assessment Who does ongoing assessment (not screening)? (total n=127)
Kindergarten teachers 100% (127)
Specialists/therapists 65% (82)
Instructional aides 34% (43)
Other (i.e Title1, Reading) 24% (30)
Other teachers 19% (24)
Administrators 10% (13)

31. Use of Assessment Data
How do you use assessment data? (all that apply) (total n=127)
Plan/adapt curriculum % (123)
Communicate with parents % (122)
Plan classroom activities % (116)
Identify children for referral to special education 91% (115)
Share with first grade teachers or others 90% (114)
Inform and complete children’s progress reports 90% (114)
Share with teachers who are working with child 89% (113)
Determine areas for more training 73% (92)
Other (i.e. Title I, to support differentiated instruction) 15% (19)

32. English Language Learners
List screening, assessment, and evaluation instruments that are used with English Language Learners. Note whether they are screening, assessment, or evaluation instruments. (total n=127)
MELA-O (Assessment) 96% (122)
Other (i.e. observation, home survey) 26% (33)
Language Assessment Scales (Assess/Eval) 20% (26)
Idea Proficiency Test (Screening) 18% (23)
Bilingual Syntax Measure (Screening/Eval) 9% (11)
MEPA % (5)
Early Screening Inventory (Screening) 3% (4)
Of 127 responses, 72 districts use only one instrument, 37 use two, and 18 use three with ELL students.

33. Educator Support
What types of support do teachers receive to implement an ongoing curriculum and assessment system? (all that apply) (total n=127)
Planning time % (122)
Time to attend training 91% (116)
Time to meet with team members 88% (112)
Time to meet with parents to review data 86% (109)
Substitutes/stipends for training/observe other programs 84% (106)
Dedicated time to summarize and analyze assessment data 56% (71)
Time to observe other programs 53% (67)
Parental recognition of the importance of assessment 44% (56)
Other (e.g. professional development., special ed and/or classroom support, team mtgs.) 12% (15)

34. Kindergarten Curriculum Guidelines
Check which tasks the district has accomplished (total n=127)
Kindergarten guidelines developed by district 98% (124)
Aligned curriculum w/MA Curriculum Frameworks 95% (120)
Aligned w/Guidelines for Preschool Learning Experiences 54% (69)
Aligned with first grade curriculum 78% (99)
Other % (24)

35. Data Systems

36. Longitudinal Data System
EEC is working with ESE to lay the groundwork for a longitudinal P-20 data system
The first step in facilitating the tracking of children’s participation in social services, education, and experiences as they progress to adulthood is the assignment of a unique identifier to each child.
This identifier allows agencies and government to evaluate the effectiveness of social service and educational programs.
ESE already has a unique identifier for school children, the SASID (state assigned student ID).
EEC is collaborating with ESE to have the SASID assigned to EEC children.
As the result of an MOU between ESE and EEC, EEC began sending data for SASID assignment to ESE starting with the teen parent population and their children as the pilot group.
Since the project began last February 2010, 39,537 records (including teen parents, teen parent children and preschoolers) have been sent to the Department of Elementary and Secondary Education (ESE). ESE has assigned new SASID to 30,069 records and has identified 1,992 records with prior SASIDs.

37. Early Childhood Information System
In FY11 EEC laid the foundation for an Early Childhood Information System (ECIS) that will provide real time data to close the state’s achievement gap, which can be analyzed by the time a child is in the 3rd grade.
The ECIS will utilize assessments to measure well-being and progress at various milestones throughout a child’s lifespan.
Implementation will include parent engagement and consent
Self-Assessment Data: EEC hopes to encourage family participation in the ECIS by enabling them to contribute their own self-assessment information, including data on relationships and environments.
Child Development Screening & Assessment Data: The ECIS will collect data on child development milestones through prescribed and tested screening and assessment tools.
Some of the tools now in use include the Ages & Stages Questionnaire, Peabody Picture Vocabulary Test (PPVT), Expressive Vocabulary Test (EVT), Social and Emotional Screening, and Woodcock-Johnson Test.
As the ECIS evolves, EEC hopes to collect assessment data through various agencies and settings where children might be receiving services.
All of these reports are available on EEC’s website under Research, Planning, and Publications

38. Early Childhood Information System
Screening and Assessment - Core Developmental and Academic Points for Data Gathering and Analysis: Five key age/development points at which to seek cross-organization information about specific children, and their families, which match recent guidance from the U.S. Department of Health and Human Services:
Birth
Age three
Age four
Age five, and
The conclusion of the 3rd grade.
The ECIS will maintain a portal for these early childhood providers to enter individual child-level scores.
The scores of only those children with parents who have provided their parental consent to share this data will be included in the child-level strength and risk analyses within the ECIS.
All of these reports are available on EEC’s website under Research, Planning, and Publications

39. The Commonwealth’s Bold Vision

40. Putting Policy Into Practice
Questions to Consider:
What is the science of formative assessment to guide child development?
How does it measure growth for children?
How do we build capacity within programs to complete screenings and use data at the child, program, and community levels?
What are the workforce opportunities and challenges in integrating formative assessment into teaching practice to improve child outcomes?
What role does screening and assessment play in engaging and communicating with families?
All of these reports are available on EEC’s website under Research, Planning, and Publications