1. Longitudinal Trends in the Performance of Scientific Peer Reviewers
Michael Callaham, MD, Charles McCulloch, PhD 
Annals of Emergency Medicine 
Volume 57, Issue 2, Pages (February 2011)
DOI: /j.annemergmed
Copyright © 2010 American College of Emergency Physicians Terms and Conditions

2. Figure 1 Scoring of the global quality score.
Annals of Emergency Medicine  , DOI: ( /j.annemergmed )
Copyright © 2010 American College of Emergency Physicians Terms and Conditions

3. Figure 2 Elements of a quality review.
Annals of Emergency Medicine  , DOI: ( /j.annemergmed )
Copyright © 2010 American College of Emergency Physicians Terms and Conditions

4. Figure 3
Distribution of individual reviewers' mean quality scores (mean of all scores by reviewer).
Annals of Emergency Medicine  , DOI: ( /j.annemergmed )
Copyright © 2010 American College of Emergency Physicians Terms and Conditions

5. Figure 4
Scores and regression slopes of sample reviewers. Four examples are illustrated of all review scores for an individual reviewer and the fitted slope of the trend (in points per year). The first and last reviewer demonstrate negative (deteriorating) slopes; the second and third, positive (improving) slopes.
Annals of Emergency Medicine  , DOI: ( /j.annemergmed )
Copyright © 2010 American College of Emergency Physicians Terms and Conditions

6. Figure 5
Distribution of individual reviewers' regression slopes (change in rating score points per year). Most reviewers are in the negative range below zero.
Annals of Emergency Medicine  , DOI: ( /j.annemergmed )
Copyright © 2010 American College of Emergency Physicians Terms and Conditions

7. Figure 6
Performance of new reviewers added each year. For the cohort of reviewers added to the pool each year, the mean number (N) in the cohort, the mean quality score of all reviews during their tenure (mean score), and slope of change during their tenure (mean slope) are plotted on the same graph for comparison. To do so, the percentage of maximum value for each variable was used (rather than the absolute number). For the number of reviewers, 100% was 254 and minimum was zero; for mean quality score, 100% was 5 and 0% was 0; for slope (change in rating score per year), 100% was (the largest change observed) and 0% (the most negative) was – The mean quality score of reviewers added in later years tended to be somewhat higher than that of their predecessors, thus maintaining a constant group average despite the decrease in slope observed for most reviewers. Despite this higher overall mean score, later reviewers' quality decreased at much the same rate as that of earlier ones.
Annals of Emergency Medicine  , DOI: ( /j.annemergmed )
Copyright © 2010 American College of Emergency Physicians Terms and Conditions