Massachusetts Universal Pre- Kindergarten Program Evaluation of the First Two Years of the Pilot Initiative Alyssa Rulf Fountain Barbara Goodson September 2008

Today’s Presentation 1.Overview of evaluation 2.Key findings 3.Policy implications 4.Next steps for evaluation 1/27/2016 2

Massachusetts UPK Goal of UPK: to promote school readiness and positive outcomes for children, especially at risk Pathway to outcomes: through universally accessible, quality early education and care UPK being enacted through mixed service delivery system 1/27/2016 3

UPK Pilot Initiative: Implemented FY 2007 Grants awarded to selected eligible providers to enhance quality of care Grants were targeted to settings that already showed commitment to quality Grantees in the evaluation received grants in both FY07 and FY08 1/27/2016 4

UPK Classroom Quality Grants: FY07 and FY08 Evaluation includes the 68 agencies and 125 program sites (centers, public school preschool programs, and family child care providers) that received Classroom Quality grants in FY07 – Child care centers: 52 agencies/81 centers – School districts: 5 school district program sites – Family child care: 9 agencies/39 providers In FY08, an additional 87 program sites received Classroom Quality grants (not included in this evaluation) 1/27/2016 5

Evaluation of UPK Pilot Program Current evaluation represents first step in assessing UPK – Focus on implementation process – Measurement of most immediate outcomes— use of funds, perception of impacts on quality Next steps for evaluation – Measuring longer-term outcomes on quality and child outcomes 1/27/2016 6

Research questions – Characteristics of grant recipients and children they serve – How grantees allocated grant funding across allowable expenditure areas – Grantee/provider satisfaction with grant program – Areas where greatest program needs remain Interviewed all program administrators and a sample of teachers and providers Summary of Evaluation Design 1/27/2016 7

Summary Findings: Implementation of Pilot initiative Pilot grant program being implemented as planned Agency satisfaction with grant program – High marks for targeting of grant funds to areas linked to quality – High marks on perceived positive effects of grants on quality of care 1/27/2016 8

Abt Associates Inc. Allocation of UPK Grants by Year—Types of Expenditures for All Grantees UPK Grant Fund Allocations: Any Grant Funds by Year

Types of Expenditures by Grantee Type UPK Grant Fund Allocations: Any Grant Funds by Grantee Type (2008 only)

Abt Associates Inc. Allocation of UPK Grants by Year—% of Fund Expenditures for All Grantees UPK Grant Fund Allocation: % Grant Funds by Year

Abt Associates Inc. % Expenditures by Grantee Type UPK Grant Fund Allocation: % Grant Funds by Grantee Type (2008 only)

Summary Findings: Presence of At-Risk Children in Grantee Program Sites Sixty four percent of children receiving financial assistance Half of children from minority groups 1/3 of children from non-English language homes 15% of children identified with special needs Proportion at-risk children (low income, minority language/cultural groups) varies by type of care, with more at-risk children in family child care 1/27/

Abt Associates Inc. Race/Ethnicity of Children in UPK Programs—All Grantees Abt Associates Inc. Race/Ethnicity of Children in UPK Programs—All Grantees Children in All Programs

Abt Associates Inc. Race/Ethnicity of Children in UPK Programs by Grantee Type Abt Associates Inc. Race/Ethnicity of Children in UPK Programs by Grantee Type Children in Center-based Programs Children in Family Child Care Homes White Hispanic Black, non-Hispanic Other Children in Public School Programs

Abt Associates Inc. Classroom teachers  Average = 14 years early childhood experience  18% had an associate’s degree  6% had a CDA  47% had a bachelor’s degree  29% had a graduate degree Family child care providers  Average = 18 years of early childhood experience  8% had an associate’s degree  67% had a CDA  25% had a bachelor’s degree UPK Teacher/Provider Backgrounds

Grantee Views of Remaining Needs (Beyond Current Grant Funding) Program sites – Most pressing need identified by majority of grantees: Staff compensation – Half of child care center programs also see need for professional development Family child care agencies – Wider range of needs identified – Needs identified by more than half of agencies: professional development, comprehensive services (60%) – Half of agencies also see need for curriculum/ educational materials and staff compensation 1/27/

Overview of Policy Implications Need strategies to improve quality of care for all program sites, regardless of starting point Develop strategies for promoting quality in different types of care settings Refine allowable uses of grant funds, especially by same sites over time Bring parents into the equation 1/27/

Policy Implications: Strategies to Improve Quality of Care Need two-pronged approach – Research about effecting child outcomes supports targeting quality to achieve excellence – Need simultaneous support for lower quality programs Need to consider providing more quality- related technical assistance/training to ensure maximum impact on quality--programs can’t do it alone 1/27/

Abt Associates Inc. Policy Implications: Improving Quality in Different Types of Care Settings Family child care grantees present unique challenges  Agencies focus on materials even with time for planning  Providers more removed from grant program  High priority because of presence of high % of at-risk children/children from minority language and cultural groups in family child care Public school programs present different challenges – School districts serve a greater number of children with special needs – Perceive a need for longer days with children 1/27/

Abt Associates Inc. Policy Implications: Guidance on Use of Grant Funds Possible changes in allowable areas of spending to target areas with closest links to child outcomes – Consider becoming more prescriptive/ narrowing allowable fund uses – Possibility of tightening spending uses over time, leaving broader in first years and narrowing allowable uses in later years 1/27/

Abt Associates Inc. Policy Implications: Bringing in Parents UPK program currently low-profile for public Parents need to play a more critical role as the program grows – Programs need help with parent materials to explain program and its relevance to families, children – Parent choices still mostly driven by personal considerations (cost, location, hours, etc). Quality considerations often secondary. 1/27/

Abt Associates Inc. Next Steps in Evaluation: Assessing Program Quality Looking at longer-term outcomes for early care and education system Level of quality of care in all programs and separately for UPK, UPK-eligible settings – Directly measure program quality in sample of including all types of settings Accessibility of quality care for all children but especially at-risk children – Examine, for sample of communities, the supply of UPK and UPK-eligible care settings compared with demand 1/27/

Abt Associates Inc. Next Steps in Evaluation: Descriptive Study of Child Outcomes Assess skills in early literacy, math, social adaptation, self-regulation Assess children in random sample of settings Stratify by type of care settings, UPK status Assess at multiple points in time to create longitudinal picture over time – Beginning & end of Pre-K, end of kindergarten & grade 1 Continue assessments over multiple years, to establish trends over time 1/27/

Abt Associates Inc. Measuring Child Outcomes: What This Can Tell Us Status of MA children over time vs national norms Status of subgroups of children – Children in UPK settings, UPK-eligible settings, other settings – Children in different types of settings – At-risk children 1/27/