1. American Evaluation Association EVALUATION 2009 November 14, 2009 Building Data Systems to Support Evaluation in a Biomedical Research and Development (R&D) Environment Multipaper Session 736

2. Session Participants James Corrigan Chair Linda Kupfer CareerTrac - An aid to Effective Evaluation Kevin Wright The Electronic Scientific Portfolio Assistant (e-SPA): A Tool for Analysis of NIH Project Portfolios James Onken Data Infrastructure and Access: The National Institutes of Health Research Portfolio Online Reporting Tool (RePORT) Robin Wagner Discussant

3. National Institutes of Health Office of the Director National Cancer Institute National Eye Institute National Heart, Lung and Blood Institute National Human Genome Research Institute National Institute on Aging National Institute on Alcohol Abuse and Alcoholism National Institute of Allergy and Infectious Diseases National Institute of Arthritis & Musculoskeletal & Skin Diseases National Institute of Biomedical Imaging and Bioengineering National Institute of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Institute of Dental and Craniofacial Research National Institute of Diabetes & Digestive & Kidney Diseases National Institute on Drug Abuse National Institute of Environmental Health Sciences National Institute of General Medical Sciences National Institute of Mental Health National Institute of Neurological Disorders and Stroke Fogarty International Center National Library of Medicine National Institute of Nursing Research National Center for Complementary & Alternative Medicine National Center on Minority Health & Health Disparities National Center For Research Resources Clinical Center Center for Information Technology Center for Scientific Review

4. About the National Institutes of Health (NIH) Primary US agency for supporting and conducting biomedical research Invests > $30B in biomedical research > 80% funding awarded through nearly 50,000 competitive grants to researchers at 3,000 research institutions About 10% funding supports 6,000 intramural researchers

5. Systems Supporting Internal Decision-Making Providing timely data and analyses in support of evidence-based decisions Tools supporting monitoring, analysis, and evaluation of NIH-sponsored research portfolios at the portfolio, project, and researcher levels Examples presented: CareerTrac; e-SPA Analyses performed with these systems have the potential to be incorporated into publicly available electronic resources

6. Enterprise-Wide Public Systems For large scientific enterprises, the aggregation of data across many and varied sponsored programs and projects is a significant challenge in multiple aspects: Data collection Aggregation Analysis Presentation/access An example of standard reports and analyses that integrate information on research activities from across the 27 institutes and centers of NIH will be presented: NIH RePORT Website An example that integrates information on research activities from across the 27 institutes and centers of NIH will be presented: NIH RePORT Expenditures and Results (RePORTER) system

7. Sample Questions Addressed What are the career outcomes of individuals trained through Fogarty International Center programs? [CareerTrac] What is the quality of the research supported through NIH infectious disease programs? [e-SPA] What are the trends in characteristics of the biomedical research workforce? [RePORT] What projects did NIH fund in 2008 in the area of… and what findings have been published? [RePORTER]

8. 8 NIH Reporting Data Systems Program Information e-SPA RePORT Database CareerTrac External Databases Research Results Projects and People FIRST NIH Guide USPTO RCDC IMPAC II