1. IES Grant Writing Workshop for Efficacy and Replication Projects
Joan E. McLaughlin, Ph.D.
Deputy Commissioner
National Center for Special Education Research

2. Overview of Presentation
Structure of the Institute of Education Sciences
Research Topics within NCSER & NCER
Goals within Research Topics
Research Narrative for Goal 3 applications
Preparing and Submitting an Application
What Happens Next?
Questions

3. Structure of the Institute of Education Sciences

4. IES Organizational Structure
Office of the Director
National Board for Education Sciences
National Center for Education Research
National Center for Education Statistics
National Center for Education Evaluation
National Center for Special Education Research

5. FY 2012 Research Grant Topics in NCSER and NCER

6. Special Education Research Topics (84.324A)
Early Intervention and Early Learning in Special Education
Reading, Writing, and Language Development
Mathematics and Science Education
Social and Behavioral Outcomes to Support Learning
Transition Outcomes for Special Education Secondary Students
Cognition and Student Learning in Special Education
Professional Development for Teachers and Related Service Providers
Special Education Policy, Finance, and Systems
Autism Spectrum Disorders
Technology for Special Education
Families with Children with Disabilities

7. Education Research Topics (84.305A)
Reading and Writing
Mathematics and Science Education
Cognition and Student Learning
Social and Behavioral Context for Academic Learning
Education Technology
Effective Teachers and Effective Teaching
Improving Education Systems: Policies, Organization, Management, and Leadership
Postsecondary and Adult Education
Early Learning Programs and Policies
English Learners

8. Research Goals within Topics

9. Solution Driven Research
Research intended to contribute to the solution of practical education problems:
Exploration Goal One
Development and Innovation Goal Two
Efficacy and Replication Goal Three
Scale-up Goal Four
Measurement Goal Five

10. Exploration
Explore the association between malleable factors and education outcomes
A malleable factor can be changed by the education system be it a characteristic of students (e.g., skills, behaviors), teachers (e.g., credentials, practices) or school (e.g., climate, size), or an education program or policy
Underlying processes that enhance or inhibit learning
Aspects of a school, district, or community associated with beneficial education outcomes
Education interventions associated with beneficial education outcomes (e.g., professional development, curricula, policies)
Explore factors that mediate or moderate the relationship between malleable factors and student outcomes

11. Development and Innovation
Develop new interventions (e.g., instructional practices, curricula, teacher professional development)
Demonstrate the feasibility of the intervention for implementation in an authentic education delivery setting
Collect pilot data on promise of intervention to achieve intended outcomes

12. Efficacy and Replication
Causal test of whether or not a fully developed intervention has a beneficial impact on student outcomes relative to a counterfactual in an authentic educational setting
Takes place under “ideal” conditions
Homogenous sample of students/schools
Extra assistance to support high implementation fidelity

13. Scale-up Evaluation
Independent causal test of whether or not a fully developed efficacious intervention has a beneficial impact on student outcomes relative to a counterfactual in an authentic educational setting under routine implementation
Independent: evaluation team has no financial interest in intervention
Efficacious: evidence of intervention’s efficacy
Routine implementation: as implemented by practitioners with expected level of support if adopted by a school or district

14. Goal 5: Measurement
Develop and validate assessments or other measurement tools 14

15. Overview of Efficacy and Replication Grant Applications

16. Efficacy
Efficacy is the degree to which an intervention has a net positive impact on the outcomes of interest relative to the program/practice to which it is being compared

17. Replication
Replication projects determine if an intervention that has produced a positive impact under one set of conditions will produce a positive impact under different conditions (e.g., the same intervention with different populations of students, different educational settings, or different levels of support)

18. Purpose of Efficacy and Replication Projects
To estimate the strength or potency of the impact of the intervention
To inform the degree to which the intervention can be feasibly or practically implemented
To assess implementation fidelity

19. Appropriate Interventions for Efficacy and Replication Projects
Interventions that are widely used but have not been rigorously tested
Interventions that are fully developed, have evidence of their feasibility and promise, but are not widely used
Interventions that have a positive effect under one set of conditions

20. Efficacy and Replication Projects
Use experimental or quasi-experimental research designs that minimize selection bias or allow it to be modeled
Employ a limited set of conditions
Often involve more support from the intervention developer/researcher than is typically available

21. What percentage of research grants are Efficacy and Replication projects?
22 to 23 percent of the research projects funded by IES are Efficacy and Replication projects

22. Efficacy and Replication Applications
Research Narrative

23. Components of the Research Narrative
Significance
Research Plan
Personnel
Resources

24. Significance Describe fully-developed intervention
Provide rationale for why testing the efficacy of this intervention is important
Present a clear theory of change to guide the understanding of the intervention and expected outcomes

25. Research Plan (1) Clearly articulate all research questions
Does the intervention lead to a meaningful improvement in education outcomes?
What are the conditions that support or hinder the implementation of the intervention?
What processes mediate the relation between the intervention and student outcomes?
Make it clear that we expect applicants to do more than pose the causal question

26. Research Plan (2)
Define sample (include size requirements, exclusion rules, and discussion of attrition issues)

27. Research Plan (3) Research Design
Use a randomized controlled trial (RCT) whenever possible
Clearly justify the use a quasi-experimental design and address internal validity threats

28. Research Plan (4) Describe the power analysis
- Provide rationale for assumptions
- If subgroups are integral to your study, consider whether you have sufficient power to address the relevant research questions

29. Research Plan (5)
Provide detailed description of the chosen measures (include information on appropriateness, reliability, validity)
Don’t forget to detail any researcher-developed measures
Provide clear description of data collection procedures and timing of data collections

30. Research Plan (6) Describe how fidelity will be assessed
Clearly describe the comparison condition
Identify mediating and moderating variables and how they will be collected
Detail the plan for analysis of qualitative and quantitative data

31. Personnel Content domain experts Methodological expertise
Statistical expert
Staff experienced in conducting research in chosen setting and with population

32. Personnel (2) Qualifications Roles and Responsibilities
Percent of time devoted to the project
DON’T FORGET to address the issue of the objectivity of the evaluation if the intervention developer is part of the team

33. Resources Address institutional resources
Document the support of those in the research setting (e.g., letters from school districts or schools, preschools)

34. Efficacy and Replication
Typical award: $250,000 to $650,000 per year
Maximum of 4 years and $3,500,000 34

35. Efficacy Follow-up Studies
Examine the sustainability of the impacts of an intervention after the original study on:
- students that received the intervention and have moved on
- a new group of students in the same setting when additional resources are no longer provided for the intervention

36. Efficacy Follow-up Studies
Emphasis in narrative on previous study, including sample and effects
Typical award: 150,000 to $300,000 per year
Maximum of 3 years and $1,200,000

37. Preparing and Submitting an Application

38. Getting Started Request for Applications
IES Grants.gov Application Submission Guide
Application Package

39. Relevant Requests for Applications (RFAs)
Education Research Grant Program (84.305A)
Special Education Research Grant Program (84.324A)

40. Documents on the IES Website
FY 2012 Requests for Applications and the Grant Submission Guide are available on:
Sign up for the IES Newsflash:

42. Finding Application Packages
FY 2012 Application Packages are available on
June Application Package available on April 21, 2011
September Application Package available on July 21, 2011

44. Letters of Intent For June competition, due 4/21/2011
For September competition, due 7/21/2011
Non-binding, but strongly encouraged
Submitted electronically
Instructions at iesreview.ed.gov
Receipt via

45. IES assistance Read the Request for Applications carefully.
Call or IES program officers early in the process.
As they have time, IES program staff will review draft proposals and provide feedback to applicants.

46. Follow Directions and Attend to Details
Read Program Announcements and Application Instructions carefully
Eligible Applicants
Special Requirements
Content and Formatting Requirements
Application Submission and Processing
Application Due Dates

47. Application Due Dates in 2011
Education and Special Education Research:
June 23 and September 22
At 4:30:00 p.m. (EASTERN)

48. What Happens Next?

49. Peer Review Proposal is reviewed for compliance.
Compliant proposals are assigned to a review panel.
For Goal 3, three panel members conduct primary review of each application.
Most competitive applications are reviewed by full panel at the panel meeting.

50. Notification
All applicants will receive notification of the status of their application.
All applicants receive copies of reviewer comments.
Applicants who do not receive an award should consider revising and resubmitting their application. Applicants are encouraged to contact the program officer about the resubmission.

51. Questions

52. Frequently Asked Questions
Advantage of June vs September submission? No.
Possible start dates?
For June submission: 3/1/12 – 9/1/12
For September submission: 7/1/12- 9/1/12

53. Frequently Asked Questions (2)
Can I plan to resubmit in the next competition if I don’t get funded? There is generally not enough time for you to get the reviews, attend to the comments and resubmit.
How long does the review process take? About 8 months.

54. Special Education Research queries:
Joan McLaughlin

55. Education Research queries:
Elizabeth Albro