TIMELESS LEARNING POLICY & PRACTICE

JD HOYE President National Academy Foundation

THEORY OF CHANGE Academy Outcomes Academy Outcomes Academy Students Academy Students Local Partners & Advisory Board NAF Funders & Partners Increased Capacity to Implement NAF Model Increased Capacity to Implement NAF Model Local Education Partners NAF Supports Student Outcomes Student Outcomes Community Benefits Community Benefits

PANEL DISCUSSION Research, Data and Evaluation

Moving Toward a Performance Measurement System

Where We Have Been: Creating Standards for High Fidelity Essential Data to Collect Including all Members of the Network in the Discussion Rich Dialogue Around Performance Measurement Where We Are Going: Clarifying What it Means to Be a NAF Academy Streamline Data Collection and Reporting Self-Assessment Tool revised based on feedback from Academies and additional input from experts Increase Opportunities For Teachers, Administrators, Parents, and Students Performance Measurement

Performance Measurement System Clearly Articulated Standards & Benchmarks System for Collecting Data For Those Indicators & Benchmarks System for Analyzing Reporting, Process for Using Indicator Data to improve student performance

Affiliate Program Member Academy Leader Academy Distinguished Academy Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Stakeholders Surveys

Context for Performance Measurement System

MDRC Random Assignment Study Most Definitive No Measurable Effects – Academic Achievement – Graduation Rates – Post-Secondary Education Outcomes Positive Effects On – Student Engagement – Career Related/Work-Based Learning Experiences – Long Term Earnings (Substantial Effects) Key Findings From Previous Research

Increased Premium on Rigorous Evaluation and Reliable Estimates of Program Effects Increased Attention to Career and College Readiness and Overlap Between the Two Early Indicators of High School Success Emphasis on Data-Driven Decision-Making Key Developments in Research and Evaluation

Impacts on Academic Outcomes, Post- Secondary Education Key Drivers of Positive Impacts Understanding Cross-Site Variation in Impacts NAF Specific Impacts Need For Additional Research

What does it mean to be a NAF Academy? What Experiences Should Students Have? What Outcomes Should Fully Implemented NAF Academies Achieve? What Data Are Required – To Evaluate NAF? – To Monitor Implementation? – To Maximize program Effectiveness? Key Research & Evaluation Questions for NAF

Affiliate Program Member Academy Leader Academy Distinguished Academy Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Stakeholders Surveys

Guiding Principles: Clearly articulate a tangible rubric on what defines a NAF Academy Demonstrate emerging and best practice along the rubric Develop a tool that allows Academies to reflect on their performance against the rubric Encourage and inform an exchange of best practices and lessons learned among Academies and between NAF and the Academies Development Process: Conducted three tier process: – 280 academies reviewed – Variance review, 5 in each region – Visit 15 sites Site visits conducted by a team of experts including MDRC, National Career Academy Coalition, and members of the Data Tracking Steering Committee to the 15 Academies that scored in the Leader/ Distinguished category or are AOEs Self-Assessment Tool revised based on feedback from Academies and additional input from experts Self-Assessment and Planning Tool Overview

Data Collection & Revision Professional Development Academy Leadership The Self-Assessment Tool has 4 key sections Academy Development Advisory Board Curriculum & Instruction Work-Based Learning & Internships Each Section outlines the core concepts in the rubric… …and drives each concept to a level of tangible detail Recruitment Length of Program Provides a WBL Program Internship Completion Students’ Career Goals Integration Instructional Practices Instructional Supports Board Membership Board Function Board Supports Cohort Scheduling Academy Decision-Making Cohort scheduling ensures that academy students experience NAF academy courses plus a minimum of two core subject at each grade level. Cohort scheduling ensures that students receive instruction in a personalized environment where every child is known by a caring adult. Cohort scheduling assumes a schedule where there is weekly common planning time for the academy team of at least three staff, so that integrated learning, student supports and individualized student assessment can occur.

Affiliate Program Member Academy Leader Academy Distinguished Academy Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Stakeholders Surveys

Why Student Data? Demographic At-Risk Poverty Discipline Attendance Program NAF Academy Model Services Support Learning Student Outcomes GPA SAT High School Exit

Why ConnectEDU?

Affiliate Program Member Academy Leader Academy Distinguished Academy Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Stakeholders Surveys

Provide NAF periodic snapshots of how academies and students are performing. Provide individual academies information to help inform their own growth and improvement. Fill in the gaps in the performance measurement system with data that can not be obtained using other data sources. Why Do Surveys?

Students Teachers Parents Employers Whom To Survey?

1.Focus on a student survey to measure behavior, attitudes and outcomes for which we don’t have measures in school records. 2.Select/create instrument or instruments. 3.Select a sample of academies that represent the network. 4.Survey all students in the sampled academies. 5.Conduct in-class or on-line student survey every two years to measure changes. Surveys: Next Steps

Affiliate Program Member Academy Leader Academy Distinguished Academy Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Stakeholders Surveys

Ensure Data Quality Organize for collaborative work Build assessment literacy and skill in data interpretation Preparation

Create data overview Dig into student data Inquiry

Examine policy and procedures and instruction Plan to assess programs Acting and assessing Move Beyond Data Analysis

Affiliate Program Member Academy Leader Academy Distinguished Academy Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Stakeholders Surveys