1. Exploring the Societal Returns of Research Conducted within Hospitals New Frontiers in Evaluation Conference Vienna 2006, April 24th - 25th Antonio García Romero and Luis Royuela Morales Agencia Laín Entralgo. Consejería de Sanidad y Consumo de la Comunidad de Madrid.

2. Presentation Outline 1.Introduction 2.Framework 3.The study 4.Results 5.Conclusions

3. 1.Introduction How much to spend on medical research? It could be useful to measure the societal impacts of medical research Difficult but Necessary Difficult: we have a limited understanding of how research induce societal effects Necessary: In US: $70 billion / year saved in medical care costs $90 billion / year sales of products unrelated to health care resulted from NIH R&D Silverstein S. 1995

4. 2. Framework The evaluation of scientific research based mainly bibliometrics and expert judgement (peer review). These methods are not very useful if the objective is to measure the returns from research in terms of wealth, employment or health. Development of new indicators and methods are needed

5. 2. Framework Statement 1: Improved health is a consequence of several factors and health care quality is only one of them Health Care (hospitals) Environmental Conditions Genetics Life style decisions HEALTH

6. 2. Framework Statement 2: Health care is a result of the interaction of several factors such as research, training, health technologies and other organizational characteristics Health Care Drugs / Health Technologies Research Training

7. 2. Framework Statement 3: The research conducted within hospitals generates several societal outcomes, in addition to health care Research within hospitals Drugs innovation OtherHuman Capital Health Tech. Innovation Clinical practice improvement

8. 2. Framework Statement 4: The relationship between medical research and its societal outcomes is very complex Lewison, G From Biomedical Research to Health Improvement, Scientometrics. 54(2): 179-92.

9. 2. Framework A consequence of this complexity is the existence of a wide set of methodological approaches that can be applied to analyze this topic Macro approaches Useful to estimate aggregated effects on employment, wealth or population’s health Meso & Micro approaches Useful to estimate direct / indirect effects, policy analysis, comparative analysis

10. 2. Framework A) The Region of Madrid 6 mill. Habitants 26.000 € GDP/per capita B) The Public Health Care System of Madrid (SMS) 24 + 8 hospitals + 200 Primary Care Centers Around 10,000 physicians 2006 Budget: 6,066 mill. € C) Research in SMS 1,650 researchers (INE) 13% of their time 2,000 papers SCI / year

11. 3. The Study Objective: To analyze the effect that physicians’ research has on their clinical performance Two Assumptions: Research within hospitals should generate both scientific and societal returns research allows physicians to improve their skills and knowledge level. Methodology: Individual data (micro approach) Motivation, Satisfaction and Group Structure as explanatory variables Structural Equation Modeling

12. 3. The Study Baseline model: Comprises two sets of hypotheses Satisfaction (*) Gender SCI Ph.D Impact Motivation Group Structure Negative (-) (0: no Ph.D 1: Ph.D) Negative (-) (0: Men 1: Women) Positive (+)

13. 3. The Study Data 279 physicians-scientists working in hospitals of SMS (Madrid) Table 1. Population and sample composition. QuestionnairesNumber of Respondents (%) Number of respondents (%) (physician-scientists) Hospitals2.038410 (20,1)279 (67,8) Primary care 1.818347 (19,1)74 (21,3) Total3.856757 (19,6)352 (46,5)

14. 3. The Study Variables: 3 observed variables & 4 constructs Table 2. Definition of constructs and variables used in the study Construct/ VariableDefinition PhD1 if respondent has the PhD degree; 0 otherwise GENDERGender of the respondent (0: men; 1: women) SCIPapers SCI Journals (last 3 years) IMPConstruct for perceived impact of R&D on clinical practice SAT_CAR, SAT_CLI and SAT_SCI Constructs for job satisfaction MOTConstruct for motivation GROUPConstruct for group structure We have estimated three models Satisfaction with professional career Satisfaction with job climate Satisfaction with scientific issues

15. 4. Results Table 4. Structural models standardized coefficients Effect Model 1 SAT_CAR Model 2 SAT_CLI Model 3 SAT_SCI Satisfaction → Impact research clinical performance 0.59 (4.032) 0.64 (4.190) 0.67 (4.483) Motivation → Impact research clinical performance 0.33 (3.400) 0.30 (3.330) 0.31 (3.546) SCI→ Impact research clinical performance 0.10 (1.603) 0.14 (2.147) 0.10 (1.605) Goodness of Fit Summary χ 2 (N = 279) (1) 384.99 (p<0.001) 415.78 (p<0.001) 543.87 (p<0.001) Goodness of fit Index (GFI) (2) 0.88 0.86 Comparative Fit Index (CFI) (3) 0.920.930.92 Incremental Fit Index (IFI) (4) 0.920.930.92 (1)Non-significant χ 2 indicates that the hypothesized model fits the data (2)Similar to R 2 in regression analysis (3)CFI > 0,9 indicates a good fit (4)IFI > 0,9 indicates a good fit These cut off values are recommended by HU et al (1995)

16. 4. Results SAT_CLI Gender SCI Ph.D Impact Motivation Group Structure -0.24 -0.17 0.210.30 0.640.20 0.14

17. 5. Conclusions Motivation and Job Satisfaction have a significant and positive effect on the perceived impact of research activity on clinical performance There is a significant relationship between Research Productivity and gender, academic degree, Motivation and Group structure These results suggest the strategic value of the human factor within R&D management