1. Between-Method Differences in Prostate Specific Antigen Assays Affect Prostate Cancer Risk Prediction by Nomograms C. Stephan, K. Siemβen, H. Cammann, F. Friedersdorff, S. Deger, M. Schrader, K. Miller, M. Lein, K. Jung, and H-A. Meyer http://www.clinchem.org/cgi/content/full/57/7/995 July 2011 © Copyright 2011 by the American Association for Clinical Chemistry

2. © Copyright 2009 by the American Association for Clinical Chemistry Introduction  Artificial neural networks or logistic regression based nomograms improve prostate cancer (PCa) risk prediction - models combine tPSA, %fPSA, age, status of digital rectal examination (DRE) or prostate volume  Problem: Models were built with different patient groups using different PSA assays and PSA ranges  Aim: To evaluate effect of assay-dependent PSA and %fPSA values in various nomograms for an individual PCa prediction

3. © Copyright 2009 by the American Association for Clinical Chemistry Patients and Methods  455 PCa and 325 men with no evidence of malignancy (NEM); all histologically confirmed by 8-10 fold biopsy, PSA range 0.5-23.5 µg/L  5 PSA and free PSA (fPSA) were measured simultaneously (one with cPSA) -AxSYM (Abbott), Centaur (Siemens), Access (Beckman Coulter), Immulite (Siemens), Elecsys (Roche)  5 different nomograms using PSA, fPSA (one without fPSA), age, prostate volume were tested

4. © Copyright 2009 by the American Association for Clinical Chemistry Results  All parameters were significantly different between PCa and NEM patients (P always < 0.0001)  tPSA and %fPSA values between the five assays were almost always significantly different −only tPSA between Abbott and Siemens-c did not differ (P = 0.86)  Within every nomogram, the pairwise comparison of the predicted probabilities always showed significant differences (P <0.0001) depending on the PSA assay except for the Abbott and Siemens-c assays (see nextslide Table 1)  Nomogram IV shows the most deviant results

5. © Copyright 2009 by the American Association for Clinical Chemistry Used nomograms, predicted PCa probabilities Table 1 lists the characteristics of the five nomograms. Nomogram I is based on age, DRE and tPSA. Nomogram II uses these variables, but also includes %fPSA. Nomogram III (www.nomogram.org) was constructed by combining age, DRE, tPSA, %fPSA and sampling density. The other two nomograms include age, DRE, tPSA and fPSA (nomogram IV) and additionally TRUS and prostate volume (nomogram V). The predicted PCa probabilities are the median values of the respective individual PCa probabilities of all pts.

6. © Copyright 2009 by the American Association for Clinical Chemistry Results  Three patients, with tPSA values of approximately 2, 7, and 16 µg/L, exemplify the fact that different PCa probabilities are obtained when the results of different PSA assays are used, irrespective of the other variables (see next slide, Table 2)

7. © Copyright 2009 by the American Association for Clinical Chemistry Predicted PCa probabilities for 3 patients

8. © Copyright 2009 by the American Association for Clinical Chemistry ROC analysis  The AUC comparison for the nomograms with data from the same PSA assay revealed significantly lower AUCs (0.79–0.80) for nomogram I (without %fPSA) compared with the other nomograms (0.82– 0.87)  AUC comparison for the 5 PSA assays within the same nomogram showed clinically irrelevant differences  Contrary to AUCs, the sensitivity and specificity curves (Fig. 1, next slide) are obviously different  At chosen cutoffs (e.g. 95% sensitivity) there are large specificity differences

9. © Copyright 2009 by the American Association for Clinical Chemistry Figure 1. Sensitivity and specificity broken down by the chosen PCa probability cutoffs for five different assays in each of five nomograms showing different discrimination between PCa and NEM

10. © Copyright 2009 by the American Association for Clinical Chemistry Calibration of the nomograms  The concordance between the PCa probability predicted by the nomograms and the real (observed) rate of PCa can be visually represented in a calibration plot  With a theoretical total concordance, there is no difference between predicted probabilities and observed rates, and all points lie on the 45°line  Fig. 2 (next slide) shows the calibration differences due to assay-dependent PSA values for all nomograms  Fig. 2A shows up to 2-fold underestimation of the PCa rate regardless of the used PSA assay

11. © Copyright 2009 by the American Association for Clinical Chemistry Figure 2. Calibration plots with cubic smoothing splines for the respective observed rates and predicted PCa probabilities for 5 different assays

12. © Copyright 2009 by the American Association for Clinical Chemistry Two main conclusions 1.Results demonstrate that nomogram-based PCa prediction is influenced by the type of PSA assay that is used 2.AUC comparison alone is insufficient (almost no differences), and calibration analysis should be used for validation of models  The dependence on PSA assay calls into question the general applicability of these models without considering the suitability of a specific PSA assay for a given model

13. © Copyright 2009 by the American Association for Clinical Chemistry Thank you for participating in this month’s Clinical Chemistry Journal Club. Additional Journal Clubs are available at www.clinchem.org Follow us