# S. Garte CSR Study Section Conformity Index (SSCI) using the QVR LIKE Function.

## Presentation on theme: "S. Garte CSR Study Section Conformity Index (SSCI) using the QVR LIKE Function."— Presentation transcript:

S. Garte CSR Study Section Conformity Index (SSCI) using the QVR LIKE Function

How well do study section members react to non conforming science outside the standard concepts and approaches in their fields? The question of conformity vs. innovation in NIH peer review has been raised in the scientific and popular press. The Issue

Compute a QVR fingerprint in for all applications submitted to each study section from 2007 to 2011. (146 SRGs) Using QVR LIKE function, determine the Match Score (MS) for each application. Applications with higher match scores are more conforming to the standard applications Methodological Approach

APPLICATIONMATCH SCORECOUNCILSTATUS 1 R01 CA138852-01A111742010 01Withdrawn 1 R01 CA138852-0110812009 01Withdrawn 2 R01 CA098383-07A210152010 10Withdrawn 1 R01 CA149033-01A19612011 05Pending Award 1 R01 CA160789-019132011 05Not Discussed 1 R01 CA129436-019062007 05Withdrawn 1 R01 CA136854-018982008 10Withdrawn 1 R01 CA131319-018732007 10Withdrawn 2 R01 GM057549-09A18712006 10Awarded 1 R01 CA137056-01A18532009 10Withdrawn 1 R01 CA127478-018532007 01Withdrawn Example of Match Score Data

TWO INDEPENDENT ASSESSMENTS 1. Compare average match score for all awarded grants to all non awarded applications (t test) 2. Percentage of applications awarded in the top match score quartile compared to the lowest quartile. (Chi square) Analytical Methods

Examples of No Significant Differences between Quartiles.

PERCENT OF STUDY SECTIONS WITH POSITIVE CONFORMITY INDEX (SSCI) # SRGs (AVMS) (Quart.)(Both) (Either) CSR14511 9.0 7.6 12.4 DABP32 9.4 6.25 6.25 9.4 DBIB3525.717.114.328.6 DNDA23 8.7 8.7 8.7 8.7 DPPS28 3.6 7.1 3.6 7.1 DTCS27 3.7 3.7 3.7 3.7

Positive Correlation between SSCI and Bibliometric Measure of Quality (p < 0.001)

The great majority (90%) of CSR Study Sections do not penalize applications that do not conform The basic science SRGs tend to be more conformist than translational or clinical SRGs There is a correlation between conformity and bibliometric measure of quality. Summary and Conclusions

QUESTIONS?

No Correlation with Application Number

No Correlation with Average Match Score

No Correlation with Range of Match Scores

Download ppt "S. Garte CSR Study Section Conformity Index (SSCI) using the QVR LIKE Function."

Similar presentations