1. Appraising Evidence About Prognosis
Updated for the third edition of the Users' Guides to the Medical Literature.

2. Patient Five A’s of EBM Ask Act Acquire Apply Appraise
EBM, evidence-based medicine.
This slide returns to the evidence cycle first explained in the Education Guide “An Approach to Evidence-Based Medicine.” This Education Guide focuses on the “Appraise” step in the evidence cycle, as we appraise evidence about prognosis.
Appraise

3. Outline Introduction Users’ Guides to a study of prognosis Summary
Objectives
Definitions of key terms
Design considerations
Users’ Guides to a study of prognosis
How serious is the risk of bias?
What are the results?
How can I apply the results to patient care?
Summary 3

4. Objectives Be able to Define prognosis and prognostic factors
Evidence literacy
Recognize threats to claims that an attribute predicts future outcomes
Understand measures of outcome over time: survival curves
Evidence numeracy 4

5. Prognosis
Clinicians help patients in 3 broad ways: diagnosing or ruling out medical and health-related problems, administering treatment that does more good than harm, and giving them an indication of what the future is likely to hold

6. Prognosis Definitions
Prognosis: Prospect of survival and/or recovery from a disease as anticipated from the usual course of that disease or indicated by special features of the case
Prognostic factors: Patient or participant characteristics that confer increased or decreased risk of a positive or adverse outcome from a disease
Prognostic study: A study that enrolls patients at a point in time and follows them forward to determine frequency and timing of subsequent events

7. Prognosis Issues Possibilities (qualitative)
Which outcomes could happen?
Probabilities (quantitative)
How likely are they to happen?
Periods (temporal)
Over what time period?

8. Prognosis Uses Predictive Prescriptive Adjusted analysis
What the future is likely to hold
Prescriptive
To select treatment that does more good than harm, anticipate future state with treatment
Adjusted analysis
To deal with prognostic imbalance in a study comparing 2 management studies

9. Studies of Prognosis
Study design consideration: Strength of association
Factor
Outcome
Association

10. Studies of Prognosis
Study design consideration: Strength of association ?
Factor
Outcome
Causation

11. Studies of Prognosis Experience target outcome
Patients at risk of experiencing target event
Prognostic factor
Time
Do not experience target outcome
Cohort and case-control designs are used to explore the association between prognostic factors and outcome. If controls are used, they are patients with different prognostic factors.

12. Studies of Diagnosis Target condition present
Patients suspected of having target condition
Criterion- standard test
Diagnostic test
The diagnostic design is presented here for comparison with the prognostic design on the previous slide.
The rules of evidence for judging prognosis studies are similar to those of diagnostic studies. In a prognostic study, time is the criterion-standard test.
Target condition absent

13. Outline Introduction Users’ Guides to a study of prognosis Summary
Objectives
Definitions of key terms
Design considerations
Users’ Guides to a study of prognosis
How serious is the risk of bias?
What are the results?
How can I apply the results to patient care?
Summary 13

14. Users’ Guides: Risk of Bias
How serious is the risk of bias?
Was the sample of patients representative?
Were the patients classified into prognostically homogenous groups?
Was follow-up sufficiently complete?
Were outcome criteria objective and unbiased?

15. Risk of Bias Was the sample of patients representative?
Ideal Population
Sample Frame
Sample
Start Study
Complete Study

16. Risk of Bias Was the sample of patients representative? Expectation
People with a particular condition will, on average, experience similar outcomes as a population of similar people
Requirements
Ability to define and observe experiences in populations over time
Enough information to match people to populations

17. Risk of Bias Was the sample of patients representative?
Matching people to populations
Demographics
Age, gender, socioeconomic status, etc
Diseases
Severity, subtype
Disorders
Other illnesses or relevant conditions

18. Risk of Bias Was the sample of patients representative? Threats
Referral bias
Failure to clearly define study patients
Lack of objective criteria for defining demographics, diseases, or disorders

19. Referral Bias
Occurs when characteristics of patients differ between one setting (such as primary care) and another setting that includes only referred patients (such as secondary or tertiary care)
Example: risk of recurrent childhood seizures
1%-5% in family practice
3%-76% in neurology clinics

20. Risk of Bias Was the sample of patients representative?
Remedies to risk of bias
Report of explicit or implicit filters passed before entering study
Clear description of which patients were included and which were excluded from study

21. Risk of Bias
Were the patients classified into prognostically homogenous groups?
Expectation
Outcome for the group should be applicable to each member of the group
Requirement
Patients should be at similar point in disease process

22. Risk of Bias
Were the patients classified into prognostically homogenous groups?
Threats
Different demographics
Different diseases
Stage
Severity
Different disorders
Comorbidities that may define subgroups with different prognoses

23. Risk of Bias
Were the patients classified into prognostically homogenous groups?
Protections against risk of bias
Define and track any subgroups
Ensure same demographic, disease (stage, severity), and disorders in each prognostic group
Use clinical common sense
Have investigators missed important subgroups with different prognoses?

24. Risk of Bias Was follow-up sufficiently complete? Threats
Significant losses to follow-up
Increased likelihood that those lost have significantly different outcomes

25. Risk of Bias Was follow-up sufficiently complete?
Protections against risk of bias
Simple sensitivity analysis
Recalculate risk based on best-case scenario where all losses were free of adverse outcome
Recalculate risk based on worst-case scenario where all losses suffered the adverse outcome
Compare these recalculations to gauge the potential impact of losses

26. Risk of Bias Were study outcome criteria objective and unbiased?
As subjectivity of outcome determination increases, importance of blinding to prognostic factors increases
Impact of
Observation
Bias
Objective Subjective
Nature of Outcome of interest

27. Users’ Guides: Importance
What are the results?
How likely are the outcomes over time?
How precise are the estimates of likelihood?

28. What Are the Results? How likely are the outcomes over time?
Measures that relate events to time
Survival rate
Percent surviving at a given time
Median survival
Time at which 50% still surviving
Survival curve
Percent of original sample who have not yet had outcome of interest
Where events are discrete and time of event is precisely known

29. Life Tables
Common starting point of life table
Start of Study Patient Enters Study

30. Survival Curves
Left, Survival after myocardial infarction. Right, Results of hip replacement surgery: percentage of patients who survived without needing a new procedure (revision) after their initial hip replacement.
ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 1988;2(8607):

31. What Are the Results? How precise are the estimates of likelihood?
Confidence intervals
Range within which it is likely that true point estimate lies
Precision drops as time from exposure increases
Losses to follow-up and patients enrolled later in recruitment period

32. Survival Curve Precision
This figure shows the risk for an event by age group in children with neuroblastoma. Survival curves are more precise in the earlier follow-up periods, indicated in the figure by narrower confidence bands around the lefthand parts of the curve.
Wood LA, Coupland RW, North SA, Palmer MC. Outcome of advanced stage low grade follicular lymphomas in a population-based retrospective cohort. Cancer. 1999;85(6):

33. Users’ Guides: Applicability
How can I apply the results to patient care?
Were the study patients and their management similar to those in my practice?
Was follow-up sufficiently long?
Can I use the results in the management of patients in my practice?

34. How Can I Apply the Results?
Were the study patients and their management similar to those in my practice?
Threats
Uneven application of therapies to different subgroups
Uneven applications of therapies over time

35. How Can I Apply the Results?
Was follow-up sufficiently long?
Threats
Study follow-up too short to detect all important outcomes

36. How Can I Apply the Results?
Can I use the results in the management of patients in my practice?
Does the effect of the prognostic factor cross a decision threshold?
Reassure
Observe
Intervene

37. Outline Introduction Users’ Guides to a study of prognosis Summary
Objectives
Definitions of key terms
Design considerations
Users’ Guides to a study of prognosis
How serious is the risk of bias?
What are the results?
How can I apply the results to patient care?
Summary 37

38. Summary
The design goal of a study of prognosis is to avoid systematic overestimation or underestimation of the likelihood of outcome events in the patients under study—to make the population representative
Fore more advanced information on studies of prognosis, see How to Use an Article About Genetic Association

39. Original slides created by Robert Hayward, MD, Centre for Health Evidence
Updated by Gordon Guyatt, MD, Kate Pezalla, MA, and Annette Flanagin, RN, MA

40. Terms of Use: Users Guides to the Medical Literature Education Guides
PowerPoint Usage Guidelines
JAMAevidence users may display, download, or print out PowerPoint slides and images associated with the site for personal and educational use only. Educational use refers to classroom teaching, lectures, presentations, rounds, and other instructional activities, such as displaying, linking to, downloading, printing, and making and distributing multiple copies of said isolated materials in both print and electronic format. Users will only display, distribute, or otherwise make such PowerPoint slides and images from the applicable JAMAevidence materials available to students or other persons attending in-person presentations, lectures, rounds, or other similar instructional activities presented or given by User.
Commercial use of the PowerPoint slides and images are not permitted under this agreement. Users may modify the content of downloaded PowerPoint slides only for educational (non-commercial) use; however, the source and attribution may not be modified. Users may not otherwise copy, print, transmit, rent, lend, sell, or modify any images from JAMAevidence or modify or remove any proprietary notices contained therein, or create derivative works based on materials therefrom. They also may not disseminate any portion of the applicable JAMAevidence site subscribed to hereunder through electronic means except as outlined above, including mail lists or electronic bulletin boards.