1. Summary Document June 2011 Investing in Innovation (i3) Fund Pre-Application Meeting Note: These slides are intended as guidance only. Please refer to the official documents published in the Federal Register.

2. Note for Webinar Participants 2 The slides that will be presented during today’s pre-application meeting are available for download on the Resources page of the i3 website at: http://www2.ed.gov/programs/innovation/ resources.html

3. A Few Notes on Q&A We have budgeted time during the meeting for Q&A – Live participants should submit their questions via the computers around the room that are set up to receive questions – Webinar participants should submit their questions via the webinar chat function Due to time constraints, we will not be able to answer all questions received If you have additional questions, please send them to the i3 mailbox: i3@ed.govi3@ed.gov 3

4. A Few Notes on Q&A (cont’d) What We Can Address What We Can Address What We Cannot Address Questions about the eligibility of a specific entity Questions about the competitiveness of a specific entity or research focus Questions about 2010 applications Substantive explanation of the rationale behind inclusion or exclusion of specific items in i3 beyond what is in the Federal Register Content of the Investing in Innovation (i3) Notices and Application Timeline of the i3 program Application process 4

5. Schedule for Today 5 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

6. Investing in Innovation (i3) Fund Summary Purpose Funding Applicants To provide competitive grants to applicants with a record of improving student achievement, attainment or retention in order to expand the implementation of, and investment in, innovative practices that are demonstrated to have an impact on: Improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates Increasing college enrollment and completion rates $148.2 million (est.) to be obligated by December 31, 2011 Eligible applicants are: (1)Local educational agencies (LEAs) (2)Nonprofit organizations in partnership with (a) one or more LEAs or (b) a consortium of schools 6

7. i3 Development Validation Scale-up Types of Awards Available Under i3 Funding Available Up to $3MM/awardUp to $15MM/awardUp to $25MM/award Estimated Awards Up to 15Up to 5Up to 2 Evidence Required Reasonable - research findings or hypotheses, including related research or theories in education and other sectors Moderate – either high internal validity and moderate external validity, or vice versa Strong – both high internal validity and high external validity Scaling Required Able to further develop and scale Able to be scaled to the regional or state level Able to be scaled to the national, regional, or state level 7

8. What Makes i3 Different Builds portfolio of solutions to some of America’s most persistent educational challenges Aligns amount of funding with level of evidence Aims explicitly to scale effective programs and create a pipeline of promising innovations Provides funding for required independent evaluation in order to build understanding of “what works” 8

9. How We Think About Innovation Innovation product, process, strategy, or approach that improves significantly upon the status quo and reaches scale Innovation product, process, strategy, or approach that improves significantly upon the status quo and reaches scale Invention Baseline Scale Greater Impact Trend 9 Note: The definition of innovation on this slide is presented as an overview of the concept, not as a specific definition in the i3 program

10. Vibrant Competition in 2010 Nearly 1700 applications across all three grant categories 49 grantees - 4 Scale-up, 15 Validation, 30 Development grants – aiming to collectively serve millions of students All 49 grantees secured private-sector matching Multiple unfunded i3 applicants subsequently have identified organizations to fund at least part of their proposal 10

11. Warnings from 2010 Applicants failed to submit applications on time and so were not reviewed (this year, submit no later than 4:30:00PM DC time on August 2, 2011) Some highly-rated applicants were declared ineligible for funding because they did not meet the evidence eligibility requirements for the type of grant they requested READ THE NOTICES and FAQs UNDERSTAND THE REQUIREMENTS PLAN AHEAD 11

12. Schedule for Today 12 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

13. Major Changes from 2010 Funding by Grant Type The maximum award for each grant type has changed: Scale-up: Up to $25 million Validation: Up to $15 million Development: Up to $3 million Absolute Priorities The competition now includes five APs, with the changes noted below: Retaining: Teachers and Principals Standards and Assessments Low-Performing Schools Adding: Promoting STEM Education Improving Rural Achievement 13

14. Major Changes from 2010 Competitive Preference Priorities Applicants may identify no more than two competitive preference priorities that they wish scored. Applicants may address as many of the competitive preference priorities as they wish for the purpose of comprehensiveness. However, the Department will review and award points only for the maximum of two CPPs the applicant identifies. The competition now includes five CPPs, with the changes noted below: Retaining: Early Learning College Access and Success Students With Disabilities and Limited English Proficiency Adding: Productivity Technology 14

15. Major Changes from 2010 Selection Criteria The number of selection criteria has been reduced to 4 Specifically, selection criteria that were addressed elsewhere last year – Strength of Research, Significance of Effect, and Magnitude of Effect and Experience of the Eligible Applicant – are no longer selection criteria (but remain important parts of the competition and should be addressed by applicants) Allocation of points by selection criterion varies by competition Matching Requirements The percentage of required private sector match now differs by competition: Scale-up: 5% of the total award requested Validation: 10% of the total award requested Development: 15% of the total award requested Applicants may still request a reduction of the required match percentage 15

16. Major Changes from 2010 Limits on Grant Awards The limits on grant awards have been revised such that no grantee may receive more than one new Scale-up or Validation grant in any two-year period This is in addition to clarifying the existing limits of two new grant awards and no more than $55MM in funding in a single year This does not affect current Scale-up or Validation grantees’ opportunity to receive new Development grants or to partner on other applications 16

17. Schedule for Today 17 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

18. All Eligible Applicants Must Implement Practices, Strategies, or Programs for High-need Students 18 MUST High-need student means a student at risk of educational failure, or otherwise in need of special assistance and support, such as students who are living in poverty, who attend high- minority schools, who are far below grade level, who are over- age and under-credited, who have left school before receiving a regular high school diploma, who are at risk of not graduating with a regular high school diploma on time, who are homeless, who are in foster care, who have been incarcerated, who have disabilities, or who are limited English proficient.

19. Applicants 19 Eligible Applicants can be: 1)A local educational agency (LEA) 2)A nonprofit organization in partnership with (a) one or more LEAs or (b) a consortium of schools

20. Eligibility Requirements: LEA 20 MUST Eligible Applicant: LEA Eligible Applicant: LEA MUST, TO RECEIVE A GRANT Address one absolute priority Demonstrate that it: (a) significantly closed achievement gaps between groups of students or demonstrated success in significantly increasing academic achievement for all groups of students, and (b) made significant improvement in other areas  Establish partnerships with private sector  Secure commitment for required private sector match  Meet the evidence requirement for the type of grant for which it has applied Address one absolute priority Demonstrate that it: (a) significantly closed achievement gaps between groups of students or demonstrated success in significantly increasing academic achievement for all groups of students, and (b) made significant improvement in other areas  Establish partnerships with private sector  Secure commitment for required private sector match  Meet the evidence requirement for the type of grant for which it has applied TO RECEIVE A GRANT, MUST

21. Eligibility Requirements: Non-Profit with LEA/Consortium 21 MUST Eligible Applicant: Non-profits, in partnership with LEA(s) or a consortium of schools Eligible Applicant: Non-profits, in partnership with LEA(s) or a consortium of schools Address one absolute priority  Demonstrate that the non-profit organization has a record of significantly improving student achievement, attainment, or retention through its record of work with an LEA or schools  Secure commitment for required private sector match  Meet the evidence requirement for the type of grant for which they have applied Address one absolute priority  Demonstrate that the non-profit organization has a record of significantly improving student achievement, attainment, or retention through its record of work with an LEA or schools  Secure commitment for required private sector match  Meet the evidence requirement for the type of grant for which they have applied TO RECEIVE A GRANT, MUST

22. Key Definition: Partners 22 Official partner means any of the entities required to be part of a partnership under section 14007(a)(1)(B) of the ARRA (i.e., a non-profit organization, an LEA, or a consortium of schools). Other partner means any entity, other than the applicant and any official partner, that may be involved in a proposed project. In the case of an eligible applicant that is a partnership between a nonprofit organization and (1) one or more LEAs or (2) a consortium of schools, the partner that was the applicant, and became the grantee when the partnership was selected to receive an award, may make subgrants to one or more of the official partners Why It Is Important

23. 23 Q&A

24. Schedule for Today 24 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

25. Improve Achievement for High-Need Students Teacher and Principal Effectiveness Promoting STEM Education College- and Career-ready Standards and Assessments Improving Rural Achievement Early Learning College Access and Success Serving Students with Disabilities and Limited English Proficient Students Productivity i3 Priorities Required for all applications Must address one Absolute Priority May address up to two Competitive Preferences (0 or 1 point each) Improving Achievement in Persistently Low- performing Schools Technology 25

26. 26 Notes on Absolute Priority 1: Teacher and Principal Effectiveness “…increase the number or percentages of teachers or principals who are highly effective teachers or principals or reduce the number or percentages of teachers or principals who are ineffective, especially for teachers of high-need students…” “…by identifying, recruiting, developing, placing, rewarding, and retaining highly effective teachers or principals (or removing ineffective teachers or principals).” “…teacher or principal effectiveness should be determined through an evaluation system that is rigorous, transparent, and fair; performance should be differentiated using multiple rating categories of effectiveness; multiple measures of effectiveness should be taken into account, with data on student growth as a significant factor, and the measures should be designed and developed with teacher and principal involvement.” Two Possible Routes for Teachers or Principals Multiple Measures of Effectiveness Multiple Methods

27. 27 Notes on Absolute Priority 2 Promoting STEM Education “(a) Providing students with increased access to rigorous and engaging coursework in STEM. (b) Increasing the number and proportion of students prepared for postsecondary or graduate study and careers in STEM. (c) Increasing the opportunities for high-quality preparation of, or professional development for, teachers or other educators of STEM subjects. (d) Increasing the number of individuals from groups traditionally underrepresented in STEM, including minorities, individuals with disabilities, and women, who are provided with access to rigorous and engaging coursework in STEM or who are prepared for postsecondary or graduate study and careers in STEM. (e) Increasing the number of individuals from groups traditionally underrepresented in STEM, including minorities, individuals with disabilities, and women, who are teachers or educators of STEM subjects and have increased opportunities for high-quality preparation or professional development.” Multiple Areas of Focus Focus on Teachers or Students Focus on High-Need Populations

28. 28 Notes on Absolute Priority 3: Standards and Assessments “…standards and assessments that measure students’ progress toward college and career-readiness…” “…may include, but are not limited to, … (a)increase the success of underrepresented student populations in academically rigorous courses and programs…; (b)increase the development and use of formative assessments or interim assessments, or other performance-based tools and “metrics” that are aligned with high student content and academic achievement standards; or (c)translate the standards and information from assessments into classroom practices that meet the needs of all students, including high-need students.” “…eligible applicant must propose a project that is based on standards that are at least as rigorous as its State’s standards…” Focus on College & Career Readiness Range of Allowable Projects Rigorous Standards

29. 29 Notes on Absolute Priority 4: Persistently Low-Performing Schools “Whole-school reform, including, but not limited to, comprehensive interventions to assist, augment, or replace Investing in Innovation Fund Absolute Priority 4 schools, including the school turnaround, restart, closure, and transformation models of intervention … or …” “Targeted approaches to reform, including, but not limited to: (1)Providing more time for students to learn core academic content by expanding or augmenting the school day, school week, or school year, or by increasing instructional time for core academic subjects (2)integrating ‘‘student supports’’ into the school model to address non-academic barriers to student achievement (3)creating multiple pathways for students to earn regular high school diplomas” Projects May Choose Either Approach

30. 30 i3 Priority 4 Schools Elementary Schools Title 1 schools in corrective action or restructuring Title 1 schools in improvement and in the lowest-achieving five percent (or lowest- achieving five schools) Middle Schools Elementary school categories + Title 1 eligible schools that would be in corrective action or restructuring High Schools Elementary and middle school categories + Title 1 eligible schools with a graduation rate lower than 60% over a number of years

31. 31 Notes on Absolute Priority 5: Improving Rural Achievement “…designed to address accelerating learning and helping to improve high school graduation rates (as defined in this notice) and college enrollment rates” “…for students in rural local educational agencies” Outcomes Focused Rural Students Rural local educational agency means a local educational agency (LEA) that is eligible under the Small Rural School Achievement (SRSA) program or the Rural and Low-Income School (RLIS) program authorized under Title VI, Part B of the ESEA. Eligible applicants may determine whether a particular LEA is eligible for these programs by referring to information on the Department’s Web site at http://www2.ed.gov/nclb/freedom/local/reap.html. Multiple Metrics

32. Guidance to Applicants and Peer Reviewers on Absolute Priority 5 “Secretary encourages applicants that choose to respond to Absolute Priority 5 to also address how their applications meet one of the other Absolute Priorities” “…applicants that choose to respond to Absolute Priority 5 should identify in the application and the i3 Applicant Information Sheet all rural LEAs where the project will be implemented, or identify in the application how the applicant will choose any rural LEAs where the project will be implemented” Focus on Key Reforms Identify Rural Locations 32

33. Guidance to Applicants and Peer Reviewers on Absolute Priority 5 “…explain how the proposed innovative practices, strategies, or programs address the unique challenges of high-need students in schools within a rural LEA, resulting in accelerated learning and improved high school graduation and college enrollment rates” “Applicants may also provide information on the applicant’s experience and skills, or the experience and skills of their partners, in serving high-need students in rural LEAs in responding to Selection Criterion D. Quality of the Management Plan and Personnel” Focus on Rural Needs Demonstrate Past Experience 33

34. Note on Removing Use of Data as an Absolute Priority “For this year’s competition, the Secretary chooses not to use the priority Innovations That Improve the Use of Data (Absolute Priority 2 in the 2010 i3 NFP). This action is not intended to discourage applicants from proposing projects that improve the use of data, so long as the proposal addresses one of the absolute priorities in this notice. Specifically, proposed projects that address Absolute Priority 1--Innovations That Support Effective Teachers and Principals, Absolute Priority 3--Innovations That Complement the Implementation of High Standards and High-Quality Assessments, and Absolute Priority 4-- Innovations That Turn Around Persistently Low- Performing Schools may also include using data in innovative ways to support the broader aims of the absolute priorities” Effective Data Use Allowable in Other Priorities 34

35. Improve Achievement for High-Need Students Teacher and Principal Effectiveness Promoting STEM Education College- and Career-ready Standards and Assessments Improving Rural Achievement Early Learning College Access and Success Serving Students with Disabilities and Limited English Proficient Students Productivity i3 Priorities Required for all applications Must address one Absolute Priority May address up to two Competitive Preferences (0 or 1 point each) Improving Achievement in Persistently Low- performing Schools Technology 35

36. 36 Explanation of Competitive Preference Priorities Applicants for all types of grants may, but are not required to, identify up to two competitive preference priorities to earn extra points Points will be awarded depending on how well the applicant addresses a particular competitive preference priority, based on the judgment of the peer reviewers All competitive preference priorities are worth up to one point, and the point is awarded all or nothing by peer reviewers Important Note on CPPs The Department will not review or award points under any competitive preference priority for an application that: (1) fails to clearly identify the competitive preference priorities it wishes the Department to consider for purposes of earning the competitive preference priority points, or (2) identifies more than two competitive preference priorities

37. 37 Notes on Competitive Preference Priority 6: Early Learning “…improve educational outcomes for high- need students who are young children (birth through 3rd grade) by enhancing the quality of early learning programs” “…(a) improving young children’s school readiness (including social, emotional, and cognitive readiness) so that children are prepared for success in core academic subjects (as defined in section 9101(11) of the ESEA); (b) improving developmental milestones and standards and aligning them with appropriate outcome measures; and (c) improving alignment, collaboration, and transitions between early learning programs that serve children from birth to age three, in preschools, and in kindergarten through third grade…” Focus on High- need Children Projects Must Address All 3

38. 38 Notes on Competitive Preference Priority 7: College Access and Success “… enable kindergarten through grade 12 (K–12) students, particularly high school students, to successfully prepare for, enter, and graduate from a two- or four-year college…” “…(a) address students’ preparedness and expectations related to college; (b) help students understand issues of college affordability and the financial aid and college application processes; and (c) provide support to students from peers and knowledgeable adults.” Focus on College Graduation Projects Must Address All 3

39. 39 Notes on Competitive Preference Priority 8: Students with Disabilities & Limited English Proficiency “…address the unique learning needs of students with disabilities, including those who are assessed based on alternate academic achievement standards, or the linguistic and academic needs of limited English proficient students.” “…must provide for the implementation of particular practices, strategies, or programs that are designed to improve academic outcomes, close achievement gaps, and increase college- and career-readiness, including increasing high school graduation rates (as defined in this notice), for students with disabilities or limited English proficient students.” Focus on Either Student Population Projects That Improve Specific Outcomes

40. 40 Notes on Competitive Preference Priority 9: Improving Productivity “…applications for projects that are designed to significantly increase efficiency” “…use of time, staff, money, or other resources while improving student learning or other educational outcomes (i.e., outcome per unit of resource)” “Such projects may include innovative and sustainable uses of technology, modification of school schedules and teacher compensation systems, use of open educational resources (as defined in this notice), or other strategies.” Make Significant Improvement Must Improve Outcomes Multiple Possible Approaches

41. 41 Notes on Competitive Preference Priority 10: Technology “…projects that are designed to improve student achievement or teacher effectiveness” “…use of high-quality digital tools or materials” “…which may include preparing teachers to use the technology to improve instruction, as well as developing, implementing, or evaluating digital tools or materials” Focus on Teachers or Students Two Types of Products Multiple Possible Projects

42. 42 Q&A

43. Schedule for Today 43 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

44. Evidence in i3 44 We will provide an overview of the role of evidence in the i3 competition today The Institute of Education Sciences will be hosting a webinar(s) in coming weeks to provide greater detail on evidence and evaluation June 30 – further details will be available soon

45. Grant Types and Evidence 45 All applications must meet the evidence requirement for the type of grant they are seeking Applications that do not meet the evidence requirement will not be eligible for a grant award, regardless of scores on the selection criteria If an application is judged not to meet the evidence requirement of the grant type to which it applied, it will not be considered for a different type of i3 grant

46. Strong Evidence: Scale-up 46 Internal Validity and External Validity Evidence from previous studies whose designs can support causal conclusions (i.e., studies with high internal validity), and studies that in total include enough of the range of participants and settings to support scaling up to the State, regional, or national level (i.e., studies with high external validity) Practice, Strategy, or Program in Prior Research The same as that proposed for support under the Scale-up grant Participants and Settings in Prior Research Included the kinds of participants and settings proposed to receive the treatment under the Scale-up grant Significance of EffectEffect in prior research was statistically significant, and would be likely to be statistically significant in a sample of the size proposed for the Scale-up grant Magnitude of EffectBased on prior research, substantial and important for the target population for the Scale-up project Note: Italicized items are addressed as part of selection criterion A

47. Strong Evidence: Scale-up (cont’d) 47 Examples of Strong Evidence (1)More than one well-designed and well- implemented experimental study or well- designed and well- implemented quasi- experimental study; or (2)One large, well-designed and well- implemented randomized controlled, multisite trial

48. Moderate Evidence: Validation 48 Internal Validity and External Validity Evidence from previous studies whose designs can support causal conclusions (i.e., studies with high internal validity) but have limited generalizability (i.e., moderate external validity), or studies with high external validity but moderate internal validity Practice, Strategy, or Program in Prior Research The same as, or very similar to, that proposed for support under the Validation grant Participants and Settings in Prior Research Participants or settings may have been more limited than those proposed to receive the treatment under the Validation grant Significance of EffectEffect in prior research would be likely to be statistically significant in a sample of the size proposed for the Validation grant Magnitude of EffectBased on prior research, substantial and important, with the potential of the same for the target population for the Validation project Note: Italicized items are addressed as part of selection criterion A

49. Moderate Evidence: Validation (cont’d) 49 Examples of Moderate Evidence (1)At least one well-designed and well-implemented experimental or quasi-experimental study, with small sample sizes or other conditions of implementation or analysis that limit generalizability; (2)At least one well-designed and well-implemented experimental or quasi-experimental study that does not demonstrate equivalence between the intervention and comparison groups at program entry but that has no other major flaws related to internal validity; or (3)Correlational research with strong statistical controls for selection bias and for discerning the influence of internal factors

50. Reasonable Hypothesis: Development 50 Internal Validity and External Validity Theory and reported practice suggest the potential for efficacy for at least some participants and settings Practice, Strategy, or Program in Prior Research The same as, or similar to, that proposed for support under the Development grant Participants and Settings in Prior Research Participants or settings may have been more limited than those proposed to receive the treatment under the Development grant Significance of EffectPractice, strategy, or program warrants further study to investigate efficacy Magnitude of EffectBased on prior implementation, promising for the target population for the Development project Note: Italicized items are addressed as part of selection criterion A

51. Reasonable Hypothesis: Development (cont’d) 51 Example of Reasonable Hypothesis (1)Evidence that the proposed practice, strategy, or program, or one similar to it, has been attempted previously, albeit on a limited scale or in a limited setting, and yielded promising results that suggest that more formal and systematic study is warranted; and (2) A rationale for the proposed practice, strategy, or program that is based on research findings or reasonable hypotheses, including related research or theories in education and other sectors

52. 52 Q&A

53. Schedule for Today 53 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

54. Notes on i3 Selection Criteria and Points 54 The selection criteria are the criteria against which the peer reviewers score each application The selection criteria for each grant type have different factors The points assigned to each selection criterion vary by grant type Detailed wording for each selection criterion may be found in the Notices at the i3 website: http://www.ed.gov/programs/innovation/index.html http://www.ed.gov/programs/innovation/index.html

55. i3 Selection Criteria and Points * Depending on the number of applications received, Validation and Development grants may be judged in two tiers: all eligible applications will be scored on Criteria A, B, and D and the competitive preference priorities; then high-scoring applications will be scored on Criterion C by a different panel of reviewers. Selection CriteriaDevelopmentValidationScale- Up A.Need for the Project352530 B.Quality of the Project Design 25 30 C.Quality of the Project Evaluation 202520 D.Quality of the Management Plan and Personnel 202520 Total Points 100 55

56. Scoring Guidance for Peer Reviewers 56 Max Point Value Quality of Applicant’s Response Criterion is not addressed Criterion is poorly developed (major weaknesses) Criterion is adequately developed (some weaknesses) Criterion is well developed (minor weaknesses) Criterion is fully developed with no weaknesses 3501-1213-2324-3435 3001-1011-2021-2930 2501-89-1617-2425 2001-78-1314-1920

57. Notes on Selection Criterion A. Need for the Project 57 Extent Projects Are Exceptional Proposals for the Priority “The extent to which the proposed project represents an exceptional approach to the priority or priorities established for the competition.” Scale-up, Validation, and Development

58. Notes on Selection Criterion A. Need for the Project 58 “The magnitude of the need for the services to be provided or the activities to be carried out by the proposed project.” Scale-Up and Validation “The importance or magnitude of the effect expected to be obtained by the proposed project, including the extent to which the project will substantially and measurably improve student achievement or student growth, close achievement gaps, decrease dropout rates, increase high school graduation rates, or increase college enrollment and completion rates. The evidence in support of the importance and magnitude of the effect would be the research-based evidence provided by the eligible applicant to support the proposed project.” Scale-up and Validation Extent of the Need for the Project Substantial Expected Improvement (e.g., prior evidence, statistical significance)

59. Guidance for Scale-up and Validation Grants 59 Applicants may correct known shortcomings from prior evidence, but … Note Linking Magnitude of Effect to Presented Evidence: The Secretary notes that the extent to which the proposed project is consistent with the research evidence provided by the eligible applicant to support the proposed project is relevant to addressing the third factor of Selection Criterion A and, therefore, will be considered by the Secretary in evaluating the importance and/or magnitude of the impact expected to be obtained by the proposed project.

60. Notes on Selection Criterion A. Need for the Project 60 “The extent to which specific gaps or weaknesses in services, infrastructure, or opportunities have been identified and will be addressed by the proposed project, including the nature and magnitude of those gaps or weaknesses.” Development “The extent to which the eligible applicant demonstrates that, if funded, the proposed project likely will have a positive impact, as measured by the importance or magnitude of the effect, on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates.” Development Extent Project Develops Solutions to Specific Gaps Extent and Likelihood of Impact (e.g., prior evidence, statistical significance of research)

61. “The extent to which the proposed project has a clear set of goals and an explicit strategy, with actions that are (a) aligned with the priorities the eligible applicant is seeking to meet, and (b) expected to result in achieving the goals, objectives, and outcomes of the proposed project.” Scale-Up, Validation, and Development “The extent to which the costs are reasonable in relation to the objectives, design, and potential significance of the proposed project.” Scale-up, Validation, and Development “The potential and planning for the incorporation of project purposes, activities, or benefits into the ongoing work of the eligible applicant and any other partners at the end of the … grant.” Scale-up, Validation, and Development Notes on Selection Criterion B. Quality of the Project Design 61 Clarity of Project Goals and Strategy to Achieve Them Sustainability Designed into Project Plan Balance of Costs with Outcomes of Project

62. Notes on Selection Criterion B. Quality of the Project Design “…estimate of the cost of the proposed project, which includes the start up and operating costs per student per year (including indirect costs) for reaching the total number of students proposed to be served by the project…an estimate of the costs for the eligible applicant or others (including other partners)…” Scale-up: “…to reach 100,000, 500,000, and 1,000,000 students.” Validation and Development: “…to reach 100,000, 250,000, and 500,000 students.” “The extent to which the services to be provided by the proposed project reflect up-to-date knowledge from research and effective practice.” Scale-up and Validation Cost Effectiveness of Scaling Project to Larger Populations Balance of Proposal and Best Evidence 62

63. Guidance for All Grant Types Note: The Secretary considers cost estimates both (a) to assess the reasonableness of the costs relative to the objectives, design, and potential significance for the total number of students to be served by the proposed project, which is determined by the eligible applicant, and (b) to understand the possible costs for the eligible applicant or others (including other partners) to reach the scaling targets of [the grant type requested]. An eligible applicant is free to propose how many students it will serve under its project, and is expected to reach that number of students by the end of the grant period. The scaling targets, in contrast, are theoretical and allow peer reviewers to assess the cost-effectiveness generally of proposed projects, particularly in cases where initial investment may be required to support projects that operate at reduced cost in the future, whether implemented by the eligible applicant or any other entity. Grantees are not required to reach these numbers during the grant period. Scaling targets help assess cost-effectiveness and ARE NOT number all applicants are expected to reach 63

64. Notes on Selection Criterion C. Quality of Project Evaluation “The extent to which the methods of evaluation will provide high-quality implementation data and performance feedback, and permit periodic assessment of progress toward achieving intended outcomes.” Scale-up, Validation, and Development “The extent to which the proposed project plan includes sufficient resources to carry out the project evaluation effectively.” Scale-up, Validation, and Development Sufficient Funding to Carry Out Evaluation Understanding of Implementation and Intermediate Outcomes of Success 64

65. Notes on Selection Criterion C. Quality of Project Evaluation “The extent to which the evaluation will provide sufficient information about the key elements and approach of the project so as to facilitate replication or testing in other settings.” Scale-up and Validation “The extent to which the proposed evaluation will provide sufficient information about the key elements and approach of the project to facilitate further development, replication, or testing in other settings.” Development Evaluation Provides Information to Support Follow- on Scaling or Other Activities 65

66. Notes on Selection Criterion C. Quality of Project Evaluation “The extent to which the methods of evaluation will include a well-designed experimental study, or if a well-designed experimental study of the project is not possible, the extent to which the methods of evaluation will include a well- designed quasi-experimental design.” Scale-up “The extent to which the methods of evaluation will include a well-designed experimental study or a well-designed quasi-experimental study.” Validation Methodologies of the Evaluation 66

67. Guidance on Evaluation Goals All grantees are encouraged to: Respond to i3 performance measures Increase strength of evidence available on the impact or promise of i3-supported interventions Scale-up and Validation grantees are especially encouraged to: Produce evidence on the impact of the i3-supported intervention (reflecting any changes to the intervention or delivery model) as implemented at scale (reflecting the additional sites served and changes in types of participants and settings served) 67

68. Guidance on Evaluation Plans Applicants should present clear, detailed evaluation plans High-quality evaluation plans are encouraged to include: Key questions and proposed methods for addressing them Logic model connecting inputs with intermediate and final outcomes Sampling plan and how it will represent implementation at proposed scale Summary of data collection measures and methods Justification of budget Qualifications of proposed independent evaluation staff For experimental and quasi-experimental studies: how treatment and control/comparison groups will be formed and plan for measuring treatment/control contrast on key implementation and outcome variables 68

69. Notes on Selection Criterion D. Quality of the Management Plan “The adequacy of the management plan to achieve the objectives of the proposed project on time and within budget, including clearly defined responsibilities, timelines, and milestones for accomplishing the project tasks, as well as tasks related to the sustainability and scalability of the proposed project.” Scale-up, Validation, and Development Whether There Is a Viable Plan to Carry Out the Project 69

70. Notes on Selection Criterion D. Quality of the Management Plan “The qualifications, including relevant training and experience, of the project director and key project personnel, especially in managing large, complex, and rapidly growing projects.” Scale-up “The qualifications, including relevant training and experience, of the project director and key project personnel, especially in managing complex projects.” Validation “The qualifications, including relevant training and experience, of the project director and key project personnel, especially in managing projects of the size and scope of the proposed project.” Development Team’s Experience Leading Projects Like the One Proposed 70

71. Notes on Selection Criterion D. Quality of the Management Plan “…capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to bring the proposed project to scale on a national, regional, or State level working directly, or through partners, either during or following the end of the grant period.” Scale-up “…capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to bring the proposed project to scale on a State or regional level (as appropriate, based on the results of the proposed project) working directly, or through partners, either during or following the end of the grant period.” Validation Capacity of the Resources to Scale the Project Further 71

72. i3 Peer Review Process 72 For all three grant types… The Department will use independent peer reviewers from various backgrounds and professions who have been thoroughly screened for conflicts of interest Evaluation experts will score the selection criterion (C) focused on evidence and evaluation Peer reviewers will determine whether any competitive preference priority points should be added Validation & Development Scale-Up Only… Peer review will take place in a single tier, with three reviewers reading Selection Criteria A, B, and D, and two reviewers reading Selection Criterion C Validation & Development applications may be reviewed in a two tier process (if large numbers of applications are received) –In Tier 1, all complete applications will be reviewed and scored against Selection Criteria A, B, and D. Competitive Preference Points will also be added as appropriate by peer reviewers. –Only those highest rated in Tier 1 will advance to Tier 2, where selection criterion C will be scored

73. 73 Q&A

74. Schedule for Today 74 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

75. Key Requirements That Must Be Met Before an Award Is Made 75 The Department, before awarding i3 grants, will confirm that all eligibility requirements have been met by potential grantees, including: Requirements related to evidence –Scale-up applications must be supported by strong evidence –Validation applications must be supported by moderate evidence –Development applications must be supported by a reasonable hypothesis Evidence of an adequate match has been provided

76. Details About Evidence Eligibility Review Performed based on the What Works Clearinghouse standards for determining internal validity Experts trained in the What Works Clearinghouse standards will review the evidence submitted in support of the applicant’s claim to Strong or Moderate evidence 76

77. i3 Matching Requirement 77 Eligible applicants must obtain matching funds or in-kind donations before receiving a grant award equal to: o at least 5 percent of grant award for Scale-up o at least 10 percent of grant award for Validation o at least 15 percent of grant award for Development Only contributions from non-governmental (private) sources count towards matching requirement Contributions may be cash or in-kind Eligible applicants may count existing private sector support towards the required match so long as these funds are reallocated in support of the proposed project, and the applicant submits appropriate evidence of this commitment

78. i3 Matching Requirement 78 Pre- submission Applicants may begin to secure matching funds Applicants may indicate in their application whether they have already secured matching funds A pre-secured match will have no competitive impact As in last year’s competition, applicants that believe they will be unable to secure the private match may request a reduced matching amount by submitting a waiver request at the time of application Post - Peer Review Eligible applicants will be notified if they are “highest- rated” Highest-rated applicants will have several weeks to secure the required match amount and provide evidence of that match

79. Explanation of Limits on Grant Awards 79 Award Cap No grantee may receive more than two grant awards or more than $55 million in grant awards under this program in a single year. Additionally, no grantee may receive more than one Scale-up or Validation grant in any two-year period. Award Cap No grantee may receive more than two grant awards or more than $55 million in grant awards under this program in a single year. Additionally, no grantee may receive more than one Scale-up or Validation grant in any two-year period. Allowable Examples Scale-up ($25M) + Development ($3M) Validation ($15M) + Development ($3M) 2 Development ($3M each) Scale-up in 2010 + Development in 2011 Allowable Examples Scale-up ($25M) + Development ($3M) Validation ($15M) + Development ($3M) 2 Development ($3M each) Scale-up in 2010 + Development in 2011 Unallowable Examples 2 Scale-up or Validation Scale-up + Validation Scale-up in 2010 + Validation in 2011 3 Development ($3M each) Unallowable Examples 2 Scale-up or Validation Scale-up + Validation Scale-up in 2010 + Validation in 2011 3 Development ($3M each) Notes: Applicants with more than 2 highest-rated applications may select which 2 applications receive awards The Award Cap applies to the applicant; official partners and other partners may participate in more than 2 successful applications 2010 Scale-up or Validation grantees may receive up to 2 Development grants in 2011 Notes: Applicants with more than 2 highest-rated applications may select which 2 applications receive awards The Award Cap applies to the applicant; official partners and other partners may participate in more than 2 successful applications 2010 Scale-up or Validation grantees may receive up to 2 Development grants in 2011

80. Schedule for Today 80 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

81. Post Award Requirements 81 MUST All i3 Grantees Evaluation –Conduct an independent project evaluation* –Cooperate with technical assistance provided by the Department or its contractors –Share broadly the results of any evaluation (and data sets for Validation and Scale-up) Participate in, organize, or facilitate, as appropriate, communities of practice for the i3 program Evaluation –Conduct an independent project evaluation* –Cooperate with technical assistance provided by the Department or its contractors –Share broadly the results of any evaluation (and data sets for Validation and Scale-up) Participate in, organize, or facilitate, as appropriate, communities of practice for the i3 program MUST * Note: The quality of an applicant’s project evaluation is also a selection criterion.

82. Schedule for Today 82 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

83. Registration for Grants.gov The Grants.gov registration process involves five (5) basic steps: 1. Obtain a DUNS number 2. Register with the Central Contractor Registry (CCR) 3. Set up your Authorized Organization Representative (AOR) profile 4. Get authorized as an AOR by your organization’s e-Biz POC 5. Track your AOR status The Grants.gov registration process takes 3-14 business days to complete. You do not have to register with Grants.gov to view opportunities or to download application packages. You must register to submit. 83

84. Use the following steps to find the i3 application: Log onto www.Grants.gov Find Grant Opportunities (on the left) Basic Search In the basic search, type 84.411 and select the applicable i3 application Applicants should be careful to select the correct application from the three grant types How to Find the Application 84 Note: Applicant must download the correct version of Adobe in order to read any Grants.gov application packages

85. Click Application (across the top of the page) Click Download (towards the bottom of the page) At the bottom of the page, you will find the Application Package AND the Application Instructions. These are two separate folders and you should download both. The Application Package is a complete PDF file of all required forms. The Application Instructions contain information on submission procedures and instructions on how to complete all of the mandatory forms. You may download both folders to your desktop or some place on your computer. You may then work offline and save and submit the application when you are finished. How to Find the Application (cont.) 85

86. Important Information Once you download the application, multiple people can work on it and you can work offline Save often Once the application is complete, the “save and submit” button becomes active Submit documents as.PDF files 86

87. Forms, Assurances, and Certifications Applicants must complete all forms included in the application package. Please download and review the Application Instructions available on Grants.gov. Application for Federal Assistance (SF-424) Department of Education Supplemental Information for SF 424 Department of Education Budget Summary Form (ED 524) Sections A & B Disclosure of Lobbying Activities (SF-LLL) General Education Provisions Act (GEPA) Requirements - Section 427 Survey on Ensuring Equal Opportunity for Applicants Assurances, Non-Construction Programs (SF 424B) Grants.gov Lobbying form (formerly ED 80-0013 form) 87

88. Attach the abstract to the “ED Abstract Attachment Form” in the application package in Grants.gov. The one-page abstract, limited to 2000 characters, should include the following items: ED Abstract Form Project Title, if applicable Type of Grant Requested Absolute Priority Up to two Competitive Preference Priorities Brief project description including project activities Summary of project objectives and expected outcomes Target number of students to be served in the project Any special project features List of official and other partners 88

89. Project Narrative File(s) Project Narrative Addresses the Project Selection Criteria and the Application Content Requirements Before preparing the project narrative, interested applicants should thoroughly review the Notice Inviting Applications (NIA) for FY 2011 published in the Federal Register on June 3, 2011. The Project Narrative should include, in detail, the eligible applicant’s response to the Selection Criteria and, if applicable, the Competitive Preference Priorities. Eligible applicants should address each of the Selection Criteria since the application will be evaluated and scored against these criteria. The maximum possible score for each criterion is indicated in the NIA. Follow the formatting and page limit guidance provided in the application instructions. Please upload the project narrative as one document. 89

90. Budget Form and Related Information The ED-524 will be uploaded to the Budget Form (Listed as Department of Education Budget Information for Non-Construction Programs ED-524) Section A – Budget Summary U.S. Department of Education Funds – must include a list of the Federal funds the applicant is requesting for the proposed project by budget category and by project year Section B – Budget Summary Non-Federal Funds – should include matching funds and other non-Federal resources contributed to the project Applicants are also required to fill out Section C and provide explanations and comments necessary to interpret the information you provided on the ED-524 Sections A and B are already uploaded in Grants.gov under Mandatory Documents; however, an applicant must upload Section C under Budget Narrative Form 90

91. Budget Narrative Form Section C – Budget Narrative – must provide an itemized breakdown and justification by project year for each budget category listed in Section A for the Federal funds requested, including any procurements or subgrants. An applicant should provide a description of the sources and uses of the matching and non-Federal funds. Applicant must upload Section C – Budget Narrative. To do this, the applicant will select Budget Narrative Form under Mandatory Documents and receive a prompt to browse and upload its budget narrative. 91

92. Other Attachment File(s) Eligible applicants should attach all appendices to the Other Attachments Form. For each appendix eligible applicants are asked to save files as a.PDF, label each file with the Appendix name (e.g., Appendix A – Eligibility Requirement Checklist) and upload the file to the Other Attachments Form. Appendix A: Eligibility Requirement Checklist (LEA or Nonprofit Partnership) Appendix B: Nonprofit 501C3 status verification or Charter School status verification Appendix C: Response to Statutory Eligibility Requirements Appendix D: Response to Evidence Standards 92

93. Other Attachment File(s) (cont.) Appendix E: Waiver Request of Private Sector Match Requirement (if applicable) Appendix F: Resumes of Key Personnel Appendix G: Letters of Support and Memoranda of Understanding (if applicable) Appendix H: i3 Applicant Information Sheet Appendix I: Eligible Applicant’s list of proprietary information found in the application (if applicable) Appendix J: Other (if applicable) 93

94. i3 Program Forms i3 Eligibility Checklist An applicant applying as a local educational agency (LEA) must use the i3 LEA Eligibility Checklist An applicant applying as a partnership between a nonprofit organization and 1) one or more LEAs or 2) a consortium of schools must use the i3 Partnership Eligibility Checklist Note: An eligible applicant that is a partnership between a non-profit organization and (a) one or more LEAs or (b) a consortium of schools should provide evidence of its partnership in Appendix A following its Eligibility Checklist Download the applicable Eligibility Checklist from the i3 Web site: http://www2.ed.gov/programs/innovation/applicant.htmlhttp://www2.ed.gov/programs/innovation/applicant.html Complete, save, and upload as Appendix A – Eligibility Requirement Checklist 94

95. i3 Program Forms i3 Applicant Information Sheet Eligible applicants must complete and submit this information sheet with each application to assist ED staff in assessing the needs of the i3 competition and provide staff with a better sense of the applicant pool Applicants must fill out this form electronically. Complete, “Save As” a.PDF, and upload the completed.PDF as Appendix H – i3 Applicant Information Sheet To download the i3 Applicant Information Sheet go to the i3 Web site: http://www2.ed.gov/programs/innovation/applicant.html http://www2.ed.gov/programs/innovation/applicant.html 95

96. Submission of Proprietary Information Applicants should identify any propriety information, confidential commercial information, and financial information This will assist the Department in making any future determination regarding public release of the application List in Appendix I the page numbers on which this information can be found 96

97. Submission of Application Successful Submission Applicants should receive a time and date stamped confirmation and an assigned tracking number from Grants.gov Applicants should receive a validation email from Grants.gov within two days business days. This means the application is ready for Department pickup Applicant should receive an email with their ED assigned PR Award # (U411….) Unsuccessful Submission Applicants should receive a confirmation email with a time and date stamp and an assigned tracking number from Grants.gov If the application is received after 4:30:00 PM (DC Time) on August 2, 2011 or validation is not successful, applicant should receive an error email Email may list the error, or applicant can use their tracking number to find the submission error 97

98. Additional Requirements Limit the entire application submission to no more than 8MB Use 8.5" x 11“, white paper Front side only 1" margins at the top, bottom, and both sides Double space 12 point font or larger Use Times New Roman, Courier, Courier New, or Arial font Include a table of contents that specifies where each required part of the application is located Submit as PDF files 98

99. Problems with Grants.gov If you experience technical difficulties within Grants.gov, please contact the Grants.gov Help Desk Please keep all tracking numbers, emails, and complaints filed with Grants.gov If the problem is really a Grants.gov problem, your application may be accepted, but you must provide proof 99

100. Grants.gov contact center: http://www07.grants.gov/contactus/contactus.jsp For phone assistance call: 1-800-518-4726 www.Grants.gov 100

101. 101 Q&A

102. Schedule for Today 102 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

103. 103 i3 Key Dates Notices Published in Federal Register:June 3, 2011 Technical Assistance: Informational Conference Calls:June 2011 Pre-Application Workshops:June 17 (Washington, DC) June 24 (San Francisco) June 28 (Houston) IES Evidence & Evaluation June 30 (Time TBD) Webinar(s): Applications: Intent to Apply Due:June 23, 2011 Applications Due:August 2, 2011 Applications Reviewed:Late Summer/Fall 2011 All Grant Awards Announced: December 2011

104. Other Important Resources Investing in Innovation Fund Website: (http://www2.ed.gov/programs/innovation/index.html)http://www2.ed.gov/programs/innovation/index.html  Notices of Final Revisions to Priorities, Requirements, and Selection Criteria  Application Packages for each competition (includes the respective Notice Inviting Applications)  Eligibility Checklists  Frequently Asked Questions  Evidence Summary Table  Selection Criteria Summary Table  i3 At-A-Glance (quick reference)  Call for Peer Reviewers: http://www2.ed.gov/programs/innovation/peerreviewers.html http://www2.ed.gov/programs/innovation/peerreviewers.html All questions about i3 may be sent to i3@ed.govi3@ed.gov 104

105. Schedule for Today 105 Speaker Morning Overview of i3 ProgramJim Major Changes from 2010Jim EligibilityRon PrioritiesThelma EvidenceJefferson Break Afternoon Selection Criteria & Review ProcessJefferson Matching & Other Pre-Award RequirementsRon Post-Award RequirementsRon Putting Together an ApplicationThelma Important DatesThelma ClosingJim

106. Key Things to Remember 106 i3 will likely be extremely competitive – 1700 applications for 49 awards last year and this year we are estimating about half as many awards If you applied last year, make sure you understand the changes to the program – and be sure to address evidence and prior experience eligibility requirements Write clearly to the selection criteria: they are what the peer reviewers will use to judge your application, so explain why your project is important, what outcomes you expect, how you will achieve them, what the nation will learn Discuss how you will do what you claim you will do – do not just state that you will do it

107. Key Things to Remember 107 Evidence is critical to i3: make sure you (or your expert advisor) understand it, and if you are not sure if you meet the evidence threshold for a grant type, you may want to consider dropping down a level Check the i3 Web site often for updates to FAQs (others probably have the same question as you and we may have already answered it) and other important competition information Register for grants.gov early, make sure you understand how to use it, and leave yourself plenty of time to submit your application on time (the deadline of 4:30:00PM DC Time applies to the completion of the submission, not the beginning)