1. The Influence of Domain-Specific Metric Development on Evaluation and Design: An Example from National Institutes of Health Technology Development Programs
Panel session 1539
American Evaluation association
Evaluation 2016
October 27, 2016

2. Development of Measures to Assess NIH Technology Development Programs
Presenters
Brian Zuckerman
Development of Measures to Assess NIH Technology Development Programs
Jennifer Pohlhaus
Process and Outcome Evaluation of the NCI Innovative Molecular Analysis Technologies (IMAT) Program
Michelle Berny-Lang
Technology Program Office Perspective on Identifying Appropriate Metrics
Elizabeth Hsu
How Does Technology Development Metric Development Influence Evaluation Design?

3. National Institutes of Health
National Cancer
Institute
National Eye
National Heart,
Lung and Blood
National Human
Genome Research
National
Institute on Aging
National Institute
on Alcohol Abuse
And Alcoholism
of Diabetes &
Digestive & Kidney
Diseases
on Deafness and
Other Communication
Disorders
Eunice Kennedy Shriver
of Child Health and
Human Development
of Biomedical
Imaging and
Bioengineering
of Arthritis &
Musculoskeletal &
Skin Diseases
of Allergy and
Infectious Diseases
National Institute of
Dental and
Craniofacial Research
on Drug Abuse
Environmental
Health Sciences
of General Medical
Sciences
of Mental Health
National Institute on
Minority Health &
Health Disparities
of Neurological
Disorders and Stroke
of Nursing
Research
National Library
of Medicine
John E. Fogarty
International Center
for Advanced Study in
The Health Sciences
National Center for
Advancing
Translational Science
Complementary &
Alternative Medicine
Clinical Center
Center for
Information
Technology
Center for Scientific
Review
Office of the Director
Office of
Intramural Research
Extramural Research

4. About the National Institutes of Health (NIH)
Primary US agency for supporting and conducting biomedical research
Invests nearly $32.3B in biomedical research
>80% funding awarded through nearly 50,000 competitive grants to >300,000 researchers at >2,500 research institutions
~10% supports nearly 6,000 intramural investigators
NIH does not publicly report estimates of funding supporting technology development

5. Motivation
Explore domain-specificity with the field of research, technology, and development evaluation in the context of assessing NIH-funded projects with the primary purpose of technology development
Metrics are often developed on an evaluation-by-evaluation and program-by- program basis
Recognition of need for metrics to objectively assess a set of differing programs with similar objectives
Provide guidance on best practices/strategies for future program and evaluation design
Provide perspectives across all stakeholders in the evaluation design
Program manager
Evaluation professional developing domain-specific metrics
Evaluation professional who must implement evaluation based on proposed metrics