1. IES Funding Opportunities for Improving Education Systems Sponsored by The University Council for Educational Administration

2. Welcome & Introductions Moderator Michelle D. Young UCEA Executive Director

3. Panelists: James Benson Katina Stapleton Program Officer Program Officer National Center for Education Research Institute of Education Sciences U.S. Department of Education

4. Are You Looking for Funding for Research on Education Leadership? Research on the essential problems of schooling and leadership practice Research on improving the preparation and professional development of education leaders Research on how to improve local, state, and national education policy National Center for Education Research Institute of Education Sciences, U.S. Department of Education

5. Quick Overview: Institute of Education Sciences (IES) Research Arm of the U.S. Department of Education Mission: – Describe the condition and progress of education in the United States – Identify education practices that improve academic achievement and access to education opportunities – Evaluate the effectiveness of Federal and other education programs 5

6. IES Organizational Structure 6 National Board for Education Sciences Standards & Review Office Office of the Director National Center for Education Evaluation National Center for Education Statistics National Center for Education Research National Center for Special Education Research

7. IES Research Objectives Develop or identify education interventions (practices, programs, policies, and approaches) – that enhance academic achievement – that can be widely deployed Identify what does not work and thereby encourage innovation and further research Understand the processes that underlie the effectiveness of education interventions and the variation in their effectiveness 7

8. FY 2014 Funding for Education Leadership Research Improving Education Systems: Policies, Organization, Management, and Leadership grants program – One topic within the Education Research (84.305A) Request for Applications. Partnerships and Collaborations Focused on Problems of Practice or Policy grants program – New Request for Applications (84.305H) that includes funding for Researcher-Practitioner Partnerships Continuous Improvement Research in Education Evaluation of State and Local Programs and Policies

9. Improving Education Systems: Policies, Organization, Management, and Leadership

10. 84.305A Education Research Requires that you select a TOPIC and a GOAL Topic: Improving Education Systems: Policies, Organization, Management, and Leadership Goal: To be determined by your specific research question and research methods

11. 11 Improving Education Systems (Topic within 84.305A Education Research )

12. Improving Education Systems Middle & High School Reform Education Policy, Finance, and Systems Education Leadership Education Finance Leadership and Management Improving Education Systems: Policies, Organization, Management, and Leadership 65+ Projects since 2004 across several research programs

13. FY 2014 Improving Education Systems Supports research to improve student learning through direct improvements in the organization and management of schools and State/district education systems serving grades K through 12 establishment of policies intended to foster such improvements.

14. Focus on Education Leadership Improving Education Systems topic supports research on full range of K-12 school leaders and administrators at school, district, and State levels wide variety of school leadership roles/functions

15. Potential Education Leadership Topics Include, but are not limited to.... Variation in leadership roles by school and district context Skills and knowledge needed by school and district leaders to improve schools Professional Development Education leader recruitment and retention Licensure and certification Evaluation and assessment of leaders – Formative assessments help leaders improve practice – Summative Assessments to evaluate leadership competencies or performance for accountability purposes – Value-added evaluation systems that incorporate measures of students’ performance on standardized tests.

16. Limitations within the Improving Education Systems Topic Restricts research on professional development for aspiring K-12 leaders – Allows research on alternative certification pathways (and their components). Alternative certification pathways are defined as relatively short programs that are intended to provide intensive training to professionals and have them working in – Only allows exploratory research on other types of pre-service leadership development programs. Does not support research on education leadership at the pre-k or postsecondary levels

17. Research Must Address Student Outcomes You must include measures of student academic outcomes that are important to students, parents, teachers, and administrators. For Example: grades or achievement test scores in reading, writing, mathematics, or science; course completion and retention; high school graduation or dropout

18. Identifying Your Research Goal Exploration Development & Innovation Efficacy & Replication Effectiveness Measurement 18

19. Exploration Goal Explore associations between malleable factors and education outcomes Identify factors and conditions that may mediate or moderate relations between malleable factors and student outcomes Possible methodological approaches – Analyze secondary data – Collect primary data – Complete a meta-analysis 19

20. Development & Innovation Goal Develop an innovative intervention (e.g., curriculum, instructional approach, program, or policy) OR improve existing education interventions AND collect data on its usability, feasibility, and fidelity of implementation in actual education settings AND collect pilot data on student outcomes Development process must be iterative! 20

21. Efficacy & Replication Goal Evaluate whether or not a fully developed intervention is efficacious under limited or ideal conditions OR Gather follow-up data examining the longer-term effects of an intervention with demonstrated efficacy OR Replicate an efficacious intervention varying the original conditions OR Conduct retrospective analysis of secondary data collected in the past 21

22. Efficacy & Replication Goal The Researcher should... Ask what might be needed to implement intervention under routine practice Reduce appearance of conflict of interest for developer/evaluators Do not require confirmatory mediator analyses but recommend exploratory ones 22

23. Effectiveness Goal Evaluate whether a fully developed intervention that has evidence of efficacy is effective when implemented under typical conditions through an independent evaluation Prior to submitting an effectiveness proposal, at least two efficacy studies of the intervention with beneficial and practical impacts on student outcomes must have been completed 23

24. Effectiveness Goal IES expects researchers to: – Implement intervention under routine practice – Include evaluators independent of development/distribution – Describe strong efficacy evidence for intervention IES does not expect wide generalizability from a single study – Expects multiple Effectiveness projects to this end – Sample size is not a key distinction from Efficacy IES does not require confirmatory mediator analyses but encourages exploratory ones Cost of implementation is limited to 25% of budget 24

25. Measurement Goal Development of new assessments or refinement of existing assessments, and the validation of these assessments OR Validation of existing assessments for specific purposes, contexts, and populations 25

26. Measurement Requirements for Improving Education Systems Topic You must validate the proposed assessment against student academic outcomes that are important to students, parents, teachers, and administrators. For example: grades or achievement test scores in reading, writing, mathematics, science; Attendance, grade retention, high school graduation or dropout rates

27. Changes from Previous 84.305A See page 11 for highlights of changes in the FY 2014 RFA. Carefully read the full RFA. Applicants to all goals must describe plans for dissemination as appropriate to the proposed work. 27

28. GoalMax. Duration & Award (direct + indirect) Exploration With secondary data With primary data 2 years, $700,000 4 years, $1,600,000 Development & Innovation4 years, $1,500,000 Efficacy & Replication Follow-up study 4 years, $3,500,000 3 years, $1,200,000 Effectiveness Follow-up study 5 years, $5,000,000 3 years, $1,500,000 Measurement4 years, $1,600,000 Maximum Award Amounts (84.305A) 28

29. Partnerships & Collaborations Focused on Problems of Practice & Policy (84.305H) New Request for Applications (RFAs) to further promote research partnerships between research institutions and State and local education agencies (SEAs/LEAs) 29

30. Partnerships & IES Priorities IES seeks to… encourage education researchers to develop partnerships with state and local education agencies to advance the relevance of research and usability of its findings for day-to-day work of education practitioners and policymakers increase capacity of education practitioners to make constructive use of existing databases, conduct research, and make use of research findings to improve practice (See http://ies.ed.gov/director/board/priorities.asp)http://ies.ed.gov/director/board/priorities.asp 30

31. Partnerships & Collaborations Focused on Problems of Practice & Policy (84.305H) Researcher-Practitioner Partnerships in Education Research – Support a new or existing partnership to carry out initial research – Develop a plan for further research Continuous Improvement Research in Education – Support well-established partnerships to adapt and revise an existing approach (or approaches) using continuous improvement strategies Evaluation of State & Local Education Programs & Policies – Carry out rigorous evaluations of education programs or policies that are implemented under routine conditions by State or local education agencies to improve student academic outcomes Note: Use May 9 version posted on IES website 31

32. Education Issue addressed by 84.305H Applications Applicants may propose to address any education issue of priority to LEA/SEA The Institute is especially interested in… – School safety – Social skills, attitudes, and behaviors that contribute to student academic success – Implementation of the Common Core State Standards The peer review process will not penalize applications addressing other education issues 32

33. Requirements for all 3 R305H Topics: Student Outcomes & Student Populations All research must address education outcomes of students, including at least one of these categories: – Academic outcomes – Social and behavioral competencies that support student success in school Students from prekindergarten through postsecondary and adult education – Typically developing students, and/or – Students with disabilities or at risk for disabilities Specific requirements for identifying students at risk for disabilities status see http://ies.ed.gov/ncser/definition.asphttp://ies.ed.gov/ncser/definition.asp 33

34. Specific Requirements: Applications must be from a Partnership Partnership may be new (Topic 1 and 3) or existing (Topic 2) Applications must include at least one Principal Investigator (PI) from a research institution and at least one PI from a State or local education agency – PI from research institution: Must have the ability and capacity to conduct scientifically valid research and expertise in the education issue to be addressed – PI from State or local education agency: Must have decision-making authority for the issue within his or her agency 34

35. Specific Requirements: SEA or LEA Partner State education agencies – Examples: education agencies, departments, boards, commissions – Oversee early learning, elementary, secondary, postsecondary/higher, and adult education – Includes education agencies and tribal education agencies in U.S. territories 35

36. Specific Requirements: SEA or LEA Partner, Cont. Local education agencies which are primarily public school districts Community college districts State and city postsecondary systems – If there is a State or city higher education agency that oversees the postsecondary system, include them as an agency partner – If there is no State or city education agency that oversees the postsecondary system, the system can apply as the sole agency partner – A postsecondary system that applies as an education agency partner cannot also serve as the research institution partner in the same project 36

37. Additional Partners Partnerships may include more than one State or local education agency if they share similarities and interests Non-education State and local agencies may be partners as long as an education agency is a partner Partnerships may include more than one research institution if they have shared interests and will make unique contributions Partnerships may include other non-research organizations (e.g., issue-oriented or stakeholder groups) that will contribute to the partnership and its work 37

38. Topic 1: Research-Practitioner Partnerships in Education Research Promote joint research by research institutions and SEAs/LEAs to: – Address education issues of key importance to SEAs/LEAs – Directly contribute to SEA/LEA program and policy decisions – Provide an opportunity to develop partnership collaboration through initial research activities as well as develop a longer-term research plan Foster longer-term research partnerships to: – Provide and support the use of rigorous research- based evidence in decision-making – Continue practitioner input into research agenda 38

39. What should the partnerships do during the grant? Broadly, the partnerships should… Carry out initial research regarding the education issue, such as: – Use administrative databases to explore the relation between state or district-sponsored interventions and student outcomes – Construct on-track indicators for students and schools Develop the partnership through structured stakeholder meetings, school-site meetings and focus groups, and community forums to present objectives and findings Develop a plan for further research on the issue, including assessment of future data collection needs 39

40. Expected Products of the Partnership Grants 1.A description of the partnership as developed over the grant 2.A description of the education issue addressed by the partners 3.The results of the completed initial research 4.A plan for the partnership’s future research 5.Recommendations for how the partnership can be maintained over the longer term 6.Lessons learned from developing the partnership that could be used by others in forming such partnerships 40

41. Topic 2: Continuous Improvement Research in Education Implement, adapt, and revise existing approach(es) to addressing the education issue or problem of concern to the SEA/LEA: – The initial approach reflects knowledge and capabilities of the partnership at project start time – A short-cycle process of data collection, review and revision is used to iteratively adapt and refine implementation of the approach Pilot the revised approach to identify promise of effectiveness for its impact on student outcomes 41

42. Continuous Improvement Research in Education Foster longer-term research partnerships that will sustain collaborative research after the project is complete Foster the capacity of SEAs/LEAs to conduct research internally, including iterative, implementation research Contribute to broader understanding of how: – approaches can be adopted to address local conditions during widespread implementation; – Researchers and practitioners can collaborate to conduct implementation research that measurably improves practice 42

43. Continuous Improvement Requirements Projects to be supported under the topic are to promote joint research by partnerships of research institutions and SEAs/LEAs – Builds on an existing partnership (one year of prior work) – Addresses an education issue or problem of key importance to an SEA/LEA – Proposes a clear approach, with a theoretical and/or empirical base of support, for addressing the issue faced by the SEA/LEA – Funds research, partnership, and some training activities, NOT program costs 43

44. Expected Products of the Continuous Improvement Grant 1.Description of the partnership as developed over the grant 2.Description of the education issue addressed by the partners 3.Description of the approach used by the partnership to address the education issue 4.Description of the approach after it has been implemented, adapted and revised – Identification of the changes made in the approach, changes made in the education system, and reason for those changes 5.Description of the process used to adapt/revise the approaches along with the data summaries upon which revisions were based 44

45. Expected Products, cont. 6.Results from pilot analysis of the approach regarding evidence of the promise of the approach to improve student outcomes 7.Determination as to whether the approach is ready for a full evaluation or requires further development 8.Recommendations for how the partnership could be maintained over the longer term 9.Specific and general lessons from the revisions to the approach and changes made in the education system that improved the approach and its implementation 10.Lessons learned from the joint development work performed the partnership that could be used by other partnerships 45

46. Topic 3: Evaluation of State & Local Education Programs & Policies Promote joint evaluation research by research institutions and SEAs/LEAs – On an education program/policy of key importance to SEAs/LEAs – That will directly contribute to SEA/LEA program and policy decisions – Provide an opportunity to develop the partnership through the evaluation Foster longer-term research partnerships – Provide and support the use of rigorous research- based evidence in decision making – Continue practitioner input into research agenda 46

47. What should the partnerships do during the grant? Broadly, the partnerships should…  Identify an education program or policy implemented by a SEA/LEA, and of high priority to that agency, intended to improve student achievement  Carry out an evaluation of that program/policy 47

48. Expected Products of the Grants Causal evidence of the impact of a clearly specified program/policy implemented by a SEA/LEA Conclusions on and revisions to the theory of change that guides the program/policy – Contributions to our theoretical understanding of education processes and procedures 48

49. Expected Products of the Grants, cont. If a beneficial impact is found… – The organizational supports, tools, and procedures needed for sufficient implementation of the core components of the program/policy under routine practice should be identified If a beneficial impact is not found… – A determination should be made whether and what type of further research would be useful to revise the program/policy and/or its implementation The financial costs of the program/policy 49

50. Award Parameters (84.305B-H) ProgramMax. Duration Max. Award (direct + indirect) 305H Researcher-Practitioner Partnerships 305H Continuous Improvement Research 305H State/Local 2 years 4 years 5 years $400,000 $2,500,000 $5,000,000 50

51. Application Submission & Review Process “Because of inevitable electronic glitches, starting as early as possible in developing the grant application, as well as submitting it electronically is critical.” Successful Grant Writing

52. Application Deadline Application Package Posted & Submission Guide Available Start Dates Sept 4, 2013 4:30:00 PM DC Time June 6, 2013July 1, 2014 to Sept 1, 2014 Important Dates and Deadlines 52

53. Review Application Requirements  Request for Applications  Currently available at http://ies.ed.gov/fundinghttp://ies.ed.gov/funding  Grants.gov Application Submission Guide  Currently available at http://ies.ed.gov/fundinghttp://ies.ed.gov/funding  Application Package  Currently available at Grants.gov 53

54. Sign-Up for the Application Process Webinar During this webinar, IES staff will provide information regarding the grant submission process. The webinar will focus on information in the application instructions package, including content and formatting requirements, human subjects clearance, and application forms. Tuesday, July 23, 2013, 2:00PM – 3:30PM ET Register at http://ies.ed.gov/funding/webinars/.http://ies.ed.gov/funding/webinars/

55. Peer Review (Standards & Review Office) Compliance screening for format requirements Responsiveness screening for program requirements Assigned to review panel – 2-3 reviewers (substantive & methodological) – If scores are strong enough, application is reviewed by full panel Many panelists will be generalists to your topic There will an expert in every procedure you use – Panel provides an overall score plus scores on Significance, Research Plan, Personnel, and Resources 55

56. Notification Process 56 All applicants will receive e-mail notification that the following information is available via the Applicant Notification System (ANS): Status of award Reviewer summary statements If you are not granted an award the first time, plan on resubmitting, and talk to your Program Officer

57. Help Us Help You Read the Request for Applications carefully Call or e-mail IES Program Officers early in the process As time permits, IES program staff can review draft proposals and provide feedback Don’t be afraid to contact us! 57

58. For More Information http://ies.ed.gov/funding Katina Stapleton Katina.Stapleton@ed.gov James Benson James.Benson@ed.gov 58

59. Q & A

60. Thank you for your attendance and participation. This session will be available on the UCEA website: www.ucea.org