1. The “Secrets” to Securing IES Funding: Some Lessons Learned as an IES Standing Panel Member Geoffrey D. Borman Professor, Educational Leadership and Policy Analysis, Educational Psychology, and Educational Policy Studies

2. Types of Grants Goal 1: Problem Identification Goal 2: Development Goal 3: Efficacy Trials Goal 4: Scale-up Effectiveness Trials Goal 5: Measurement

3. How are Proposals Rated? Multiple panels to reflect the various IES funding priorities and competitions Each reviewer from the panel is assigned about 8-10 studies for review 2-3 reviewers present critiques of each proposal The remainder of the panel (including the chair) adds comments and questions Some stronger consensus is usually reached among the 2-3 primary reviewers All panel members then rate the application

4. What is Rated? Significance of the problem (1-7) Research Plan (1-7) Personnel (1-7) Resources (1-7) Budget (1-7) Overall score (5-1) Funding enthusiasm Each application evaluated on its own merit relative to an ideal

5. Overall Rating Scale Overall Score 1.0–1.5Outstanding 1.6–2.0Excellent 2.1–2.5Very Good 2.6–3.0Good 3.1–4.0Fair 4.1–5.0Poor

6. Provide Strong Theory and/or Empirical Evidence In establishing the significance of the project, theory is important Practical contributions, though, are also key The chain of empirical evidence must be established and the place of your project in that chain needs to be established and justified

7. Be Obsessively Specific Concerning Methodological Details If random assignment is proposed, describe the actual process Describe quasi-experimental methods in even more detail Describe the analytical models Describe threats to internal validity Address external validity Do a technically correct power analysis Choose good measures and defend them (IES also likes high-stakes tests)

8. Other Details It is good to have an expert on experimental design (or the design of choice) on the proposal But panels usually do not place their trust in experts— they evaluate the quality and completeness of the design Achievement is usually the main outcome of interest— ”policy-relevant” local or NCLB outcomes often favored unless specific skill measures for a domain are desired Growing interest in measuring the nature of the treatment and counterfactual to help explain null results or provide some non-experimental understandings of mechanisms driving positive results

9. Summary The rationale and significance must be strong But, the research plan generally wins or loses it Applications are not blinded, so the general reputational quality of the institution and researchers influence personnel and resource ratings a great deal Panels sometimes like “bargain” studies, but do not generally dwell on budget details If your proposal is a revise and resubmit, address the original reviewers’ concerns and describe how you are responding