Go Behind the AHRQ/NIH Study Section Door A Mock Review

The Panel Linda Greenberg, PhD Willard Manning, PhD Ming Tai-Seale, PhD

The Agenda Relevant funding mechanisms: Rs, Ks Life of a proposal Scientific review: who, where, how Critical areas for improvement Mock review Summary statement How to work with federal officials Questions and answers If they can be a speck on the paper of the proposal, we will take them through the process

Funding Mechanisms Linda

Rxx GRANTS FOR HEALTH SERVICES DISSERTATION RESEARCH (R36) R01, R03, … Support students seeking a doctorate after successful dissertation defense in areas relevant to health services research Total direct costs <$30,000 R01, R03, … Barbara?

Relevant Funding Mechanisms: K01, K02, K08, … Mentored Clinical Scientist Development Award (K08) Development of outstanding research scientists. Specialized study support for trained professionals who are committed to a career in research and have the potential to develop into independent investigators Focuses on progression to independence Study and development consistent with his/her needs, and previous research or clinical experience. The proposed length of the award must be well explained and justified Support will only be provided for the period deemed necessary to achieve independence

The Life of a Proposal Ming

Who Serve on Study Sections? Disciplines Anthropology Biostatistics Economics Epidemiology Health services research Medicine Nursing Organizational Theory Sociology Methodological Orientations Quantitative Qualitative Mixed Stages in Their Own Careers Senior Scholars Emergent scientists Everyone has equal weight in scoring, you have to communicate clearly, especially to those who don’t share your disciplinary training, You have to convince them why your proposal is good.

Where is the Review Done NIH AHRQ Let’s go there …

The Physical Setting

Protection of Human Subjects Applicant must address: Risks to human subjects Adequacy Summary reviewer choices: Human subjects NOT involved Human subjects involved, ACCEPTABLE Human subjects involved, UNACCEPTABLE Human subjects involved, exemption claimed

Inclusion of Women and Minorities Gender Code: First Character = G 1= Both Gender 2= Only Women 3= Only Men 4= Gender Unknown Third Character: A= Scientifically acceptable U= Scientifically unacceptable Minority Code: First character = M Second character: 1=Minority and Nonminority 2= Only Minority 3= Only Nonminority 4= Minority unknown Third Character: A= Scientifically acceptable U= Scientifically unacceptable

Inclusion of AHRQ Priority Populations Checklist Included Excluded Not addressed Children Elderly Rural Inner city Low income Disabled Chronically ill End of life This is only for portfolio purpose, we are not suppose to ding them on this. Adequate numbers for sub-group analysis? For excluded, including rationale?

Inclusion of Women and Minorities Gender Code: First Character = G 1= Both Gender 2= Only Women 3= Only Men 4= Gender Unknown Third Character: A= Scientifically acceptable U= Scientifically unacceptable Minority Code: First character = M Second character: 1=Minority and Nonminority 2= Only Minority 3= Only Nonminority 4= Minority unknown Third Character: A= Scientifically acceptable U= Scientifically unacceptable

Adjectives Used in Review 1.0-1.5 Outstanding 1.6-2.0 Excellent 2.1-2.5 Very Good 2.6-3.5 Good 3.6-5.0 Acceptable

Priority Score How is the summary priority score calculated? Group average Equal weight What is the fundable range? Study sections can have different norms When in doubt, ask the project officer

Critical Areas for Improvement for K0x Will

Critical Areas for Improvement for K0x It Is Not About 5 years of support 75 percent buyout $$$

It Is About Mentored Clinical Scientist Development

It Is About (cont’d) It does require mentoring It is about career development for researchers Not just about more education Not just about doing preliminary studies Don’t confuse K with series of R03’s

Disconnected Mentor Mentor’s letter not closely tied to content of proposal. Mentor’s letter written by proposer and it looks like it. Mentor approached with proposal with only week left before due date. Little impact on proposal. Worse if proposal is naïve.

Distant Mentor Always very hard to sell. Study section distrusts supposed level of commitment by mentor. Plans for linkage, visiting vague.

Who is in charge? Too many mentors No strong primary Nobody with oversight responsibility

Career Development Plan R-Avoidance Its thinly disguised research support for 5 years. Little or no education component. A La Carte Education: Lacks coherent rationale for what’s proposed. Need to lay out individual strengths and weaknesses. It’s OK to say you’re imperfect!!!

Career Development Plan Educational elements too vague Visiting Prof. Jones T times per year. T small. Lack of specificity Not clear depth of training Formal course work preferred if a good match for needs. if level appropriate Avoid lower level MPH courses.

Critical Areas for Improvement in Rs Design problem Measurement Choice of variables Intervention/comparison Analysis problem Choice of approach Technique Test

Critical Areas for Improvement in Rs Weak justification for study Background and significance unconvincing Literature review incomplete Investigator expertise deficient Needs consultants or collaborators Theoretical or conceptual model or framework Missing, deficient, or erroneous

Mock Review Chair: Willard Manning, PhD Primary: Ming Tai-Seale, PhD Secondary: Willard Manning, PhD Usually there is a tertiary reviewer K08 – Mentored Clinical Scientist Development Award

Review Guideline Candidate Career development plan Research plan Mentor/co-mentor Environment and institutional commitment Budget Human subjects Women/minorities/children SUMMARY major strengths and weaknesses Recommendation for or against funding

Summary Statement Ask Will about the story of the Pink Sheet Ming

How to Read the Pink Sheet Expect the language to be Frank, and Not overly enthusiastic Be emotionally detached, after the initial… Talk to an experienced grant-maker Resubmit unless you see “fatally flawed” Do NOT resubmit right away Recruit a “cold reviewer”

Take a Vacation …

Role of Federal Officials Linda

What Can You Expect From Project officers Read your concept paper and draft Send it in EARLY! Interpret the fundability of your priority score Scientific review administrators Assign reviewers who may have expertise to review your proposal Francis

Questions & Answers Panel

Resources Video on Peer Review for Clinical Research http://www.csr.nih.gov/Video/Video_print.asp Instructions on how to prepare your application http://grants.nih.gov/grants/funding/phs398/section_1.html Our contact information Linda: linda.greenberg@ahrq.hhs.gov Will: w-manning@uchicao.edu Ming: mtaiseale@srph.tamhsc.edu Can’t comment on any current proposal in the review process