1 EEC Board Policy and Research Committee Meeting May 5, 2014 Quality Rating and Improvement System (QRIS) Status Update

QRIS Accomplishments Developed standards Piloted standards and made revisions Launched QRIS 2012 Hired QRIS Staff - 6 Program Quality Specialists and 4 QRIS Health Advisors Began verifying QRIS applications and granting Level 2 ratings 2013 Continued application verification and granting QRIS Level 1 and 2 ratings increased collaborations with external partners and stakeholders Started comprehensive evaluation of QRIS Began development of QRIS policies and inter-rater reliability protocols 2014 Adopted new policy for Environment Rating Scales (ERS) Completed QRIS Field Survey Completed QRIS Validation Study Revisions Completed 406 classroom and provider ERS Reliable Rater assessments

Next Steps Spring 2014 Hire and train 4 PQS to fill the PQS that transitioned to licensing roles Begin granting Level 3 programs Complete QRIS evaluation and revision process- EEC Board vote in June Work with public schools developing a plan to create 4 th QRIS program type 1 st QRIS Conference for QRIS Improvement Grantees Summer 2014 Develop QRIS Policy Manual Begin granting Level 4 programs Fall 2014 Develop on-line training modules for each level of QRIS Develop standardized training modules for QRIS coaches, mentors, family systems support and other QRIS support staff Launch at-scale QRIS Validation Study Launch revised on-line application system Spring 2015 QRIS/NAEYC Alignment Study and QRIS/Head Start Alignment Study QRIS Working Group and QRIS Professional Development Review Teams- Annual QRIS Guidance Updated December 2015 QRIS Validation Study Findings Released 3

4 Although the overall average of the ECCERS-R scores was over a 5, the table indicates that a significant portion of the programs (21.5%) fell in the MINIMAL range overall. Review of the frequencies of the subscales indicates significant portions of programs falling in the minimal range for Space and Furnishings (about 31%), Personal are Routines (approximately 62%), Activities (about 49%) and Program Structure (approximately 20%). Additionally, approximately 12% of programs scored in the INADEQUATE range for Personal Care Routines, further underscoring the need for additional supports and technical assistance in this area.

5 In contrast to the preschool programs, the data indicates a greater portion of the programs falling in the minimal and inadequate ranges, both overall and among the subscales. Of particular note, around 40% of programs did not meet the GOOD benchmark overall, twice as many as with the preschool classrooms. Among the subscales, substantial portions of programs did not meet the GOOD benchmark for Space and Furnishings, Personal Care Routines, Activities and Program Structures. This suggests additional supports are needed for the Infant and Toddler programs in the promotion of quality as measured by the ITERS-R.

6 Unlike the center-based programs, a majority of the observed family child care homes did not meet the GOOD bench mark OVERALL; instead a majority fell in the minimal range. A majority of programs also fell below the GOOD range for Space and Furnishings, Personal Care Routines and Activities. Importantly, significant portion of programs fell in the INADEQUATE range for Space and Furnishings and Personal Care Routines. It is important to note that despite the large number of programs in the minimal range OVERALL, a majority of family child programs fell in the GOOD range for Language and Reasoning, Interactions, Program Structure and Parents and Staff. Despite this, a significant number of programs fell in the minimal range for Language and Reasoning (40%) and Program Structure (35%).

7 As can be seen, the preschool scores are typically higher, followed by the infant toddler programs and then the family child care homes. This pattern is found across all subscales and the OVERALL, except for the Interactions subscales in which the family child care homes score slightly higher than the infant and toddler classrooms. This suggests that more support is needed for family child care homes followed by infant and toddler classrooms to raise quality.

8 QRIS Participation Data Since the launch of the on-line, QRIS Program Manager (QPM) in January 2011, 5,329 programs (unique count) have created a total of 7,429 QRIS applications (final status) using the QPM system. April Major accomplishments: 709 of the programs participating are nationally accredited: 35 COA, 117 NAFCC and 557 NAEYC. 6,334 of those applications have been granted a QRIS Level. Level 1: 4,834 Level 2: 1,467 Level 3 pending reliable rater: 3 Level 3: 30 QRIS Program Type Central MA Metro Boston NortheastSoutheast and Cape Western MA (blank)Grand Total After School/Out of School Time Center/School Based Family Child Care Grand Total Note: In FY13, Metro (Region 4) was dissolved for Grantee purposes. Towns were reassigned to neighboring regions to align with licensing offices. 9 programs were not assigned to a region in the QPM. These are license-exempt programs that were not recognized automatically by EEC’s system.

9

Field Experiences with and Impressions of QRIS 10 The purpose of the survey was to gather input and feedback about how the QRIS is working and the ways in which the system can be improved. Random selection of programs and providers in QRIS, and high response rates, suggest the views expressed are reflective of those in QRIS more generally. UMass Donahue Institute Applied Research and Program Evaluation

Promising Findings 11 Despite some ongoing skepticism and frustration extending from early implementation, most acknowledge the inherent value in a quality improvement system and appreciate its focus on quality. Majority of QRIS participants believe the system helps programs improve, is an important priority for EEC, and will help elevate the public perception of early education and care in the state. Data show an emerging understanding of the QRIS, particularly around the system’s goals, standards, and benefits of participation. UMass Donahue Institute Applied Research and Program Evaluation

12 Most QRIS participants believe the system has helped improve their quality. - Attributed to clearer goals and improvement plans, increased self-reflection through use of tools, and self-assessment. - FCC providers were most optimistic about improvement. Most programs and providers plan to work toward the next QRIS level, many of whom expect to do so within the next year. Promising Findings Continued UMass Donahue Institute Applied Research and Program Evaluation

13 Mixed views in the field about whether QRIS ratings accurately reflect a program’s quality. Common barriers to moving up to the next level include education and training requirements, time, cost, and documentation. -Concern that progress might be hindered by just one or two standards -Education requirements were a particular concern for centers and family care providers Several respondents from public schools and family care providers felt that some aspects of the system were designed without their settings in mind. Challenges and Barriers UMass Donahue Institute Applied Research and Program Evaluation

14 Recommendations and Conclusions Develop a robust infrastructure to help programs and providers improve and advance Ensure adequate logistical and technical infrastructure is in place to accept, verify, and communicate about applications. Expand on-site support and technical assistance for programs and providers. Build programs’ and providers’ capacities to effectively use measurement tools included in the system through further training. Increase access to EEC-funded or reduced-cost college courses and trainings. UMass Donahue Institute Applied Research and Program Evaluation

15 Simplify and align QRIS with other quality measures and requirements Reduce complexity by decreasing paperwork, reducing measurement tools, and removing standards not viewed as integral to quality. Consider QRIS, NEAYC, and licensing in relation to one another. Align and streamline systems so that they can function in complementary ways. Deliberate the role of public schools in the QRIS and how the system can be aligned to best support quality improvement in school-based settings. Recommendations and Conclusions (continued) UMass Donahue Institute Applied Research and Program Evaluation

16 Remove or reduce common barriers to maximize success Address the issue of higher education requirements for centers and family care providers. Provide greater flexibility in relation to some of the standards or how the standards are met. Enhance communication and messaging to build on initial success of QRIS Develop focused messaging plans to ensure that programs and providers of all types have access to clear and consistent information about the system, its requirements, and its timelines. Build momentum and enthusiasm by capturing early success stories. Recommendations and Conclusions (continued) UMass Donahue Institute Applied Research and Program Evaluation

17 EEC Plan for Improved Information Sharing and Technical Assistance Bi-weekly QRIS announcements via subscription 2 nd Thursday /month- Technical assistance webinars for coaches and mentors 2 nd Thursday /month – Technical Assistance webinars for family child care systems support 4 th Thursday /month – Topic/content based learning for programs and providers Monthly Orientation Sessions in all EEC regional offices QRIS On-line Learning Community

18 QRIS Standards Evaluation and Revision Process EEC began a comprehensive evaluation and revision process in May Steps in this process include : 1.Review QRIS standards and measurements with our UMass Donahue partners. 2.Collaborated with RTT Tech Assistance staff to identify research that supports best practices. 3.Program Quality Unit began development of guidance documentation to inform inter- rater reliability protocols, as well made initial recommendations for standards revisions. 4.Collaborated with EEC Policy, Legal, Fiscal, I.T. and Field Operations Units. 5.Established QRIS Working Group and Professional Development Review Team. 6.Vet standards revisions with EEC staff, the field, EEC Board and other stakeholders.

Key Findings from QRIS Evaluation Process Some standards are presenting significant barriers for programs to move forward in QRIS: Programs/providers are not prepared for a health consultant visit at level 2 Lack of qualified health consultants in the field Bachelor degree requirement is unattainable for many educators and providers Ongoing Professional development does not always connect the learning experiences to certifications Inter-rater reliability protocols need to be established and maintained Programs and providers need QRIS guidance documents Advancement to the next level in QRIS after one year is not possible for some programs and providers QRIS levels need further definition 19

20 Where do we want QRIS be on January 1, 2016?

21 Statutory Requirements for a Quality Rating Improvement System (QRIS) In 2008, EEC was charged with establishing a method of measuring quality in early education and care programs. Massachusetts General Laws Part I, Title II, Chapter 15D, sec. 12 (a) The department shall establish a comprehensive system for measuring the performance and effectiveness of programs providing early education and care and services. This system shall include, but not be limited to, outcomes of the kindergarten readiness assessment system and additional educationally sound, evaluative tools or developmental screenings that are adopted by the department to assess developmental status, age-appropriate progress and school readiness of each child; outcomes of evidence-based intervention and prevention practices to reduce expulsion rates; and evaluations of overall program performance and compliance with applicable laws, standards and requirements; (e) The comprehensive system for measuring the performance and effectiveness of programs shall be designed to measure the extent to which every preschool-aged child receiving early education and care in the commonwealth through the Massachusetts universal pre-kindergarten program has a fair and full opportunity to reach such child's full developmental potential and shall maximize every child's capacity and opportunity to enter kindergarten ready to learn.

22 National QRIS Implementation 44 states have a launched a statewide QRIS system

23 Race to the Top FundingState Funding QRIS Support Staff 6 Program Quality Specialists 1 Manager of Program Quality & Improvement 4 QRIS Health Advisors 1 Child Health Care Advisor 1 Manger of QRIS Improvement Grant QRIS Improvement Grant readers and administration support Environment Rating Scales (ERS) Reliable Raters Information Technology Support 1/3 time EEC Professional Development support staff for QRIS Family Child Care Systems support staff Professional Development Educator Scholarships QRIS Measurement Tools Training Training for EEC staff and ongoing support Training and ongoing support for EPS Coach and Mentor Program and Provider Business Planning College coursework and degree attainment QRIS content related trainings Educator & Provider Support (EPS) Staff, Coaches, Mentors and Trainers Fiscal Incentives QRIS Improvement Grant – program planning and durable goods Infant/Toddler rate increase Program Quality Supports Research – QRIS Validation Study, QRIS Field Survey, support with QRIS evaluation and revisions Family Engagement QRIS Family and Community Awareness Campaign Coordinated Family and Community Engagement (CFCE) Grant Program Quality Supports

24 QRIS Staffing High QRIS caseloads- average of 833 programs:1 Program Quality Specialist Fiscal Incentives for Programs and Providers Increased salaries to compensate for advancement on the Career Ladder (QRIS requirement) Increased reimbursement to compensate FCC Providers as they further their education Staff benefits (QRIS requirement) Tiered system of reimbursement (supports the foundation of a Quality Rating and Improvement System) Information Technology Updates for the on-line application (difficult to use, slow, outdated and does not reflect all current QRIS policy) Data reporting limited and inconsistent (duplication, not well aligned with other systems) No ability to renew applications Needed Program Quality Supports

25 How do we sustain QRIS? Create a strong vision for QRIS Develop a strategic plan, including; concrete goals, objectives, timelines and key policy questions Identify strengths and challenges with the current supports Make a fiscal commitment Build an infrastructure that will sustain QRIS Strong communication with the field, families and other stakeholders Continually evaluate and revise the system to reflect needs, research and best practices

Questions, comments, concerns…