Anwar Showail.  Critical appraisal is the systematic evaluation of clinical research papers in order to establish: 1. Does this study address a clearly.

Anwar Showail

 Critical appraisal is the systematic evaluation of clinical research papers in order to establish: 1. Does this study address a clearly focused question? 2. Did the study use a valid methods to address this question? 3. Are the valid results of this study important? 4. Are these valid, important results applicable to my patient or population?

 If the answer to any of these questions is “no”, you can save yourself the trouble of reading the rest of it.

 Therapy / RCT  Systematic Review  Diagnosis  Prognosis

I. Relevance II. Validity III. Results IV. Applicability

1. Relevance  The topic addresses a common problem in my practice  POEM outcomes  Results (if valid) will change my practice

VALIDITY (RABI) Therapy Systematic Review DiagnosisPrognosis R Randomization Criteria (Inclusion and exclusion - PICOTT) Reproducibility Reference standard Representative spectrum Representative sample (Homogeneous patients with respect prognostic risk) A Allocation (concealed) Attrition Articles search Articles selection Articles abstraction Ascertained (Verification) Adjustment of prognostic factors B Blindness (3Cs) Blind Articles quality assessment Blind Comparison Blind (Outcome assessor should e blind to prognostic factors – outcome need to be Objective not subjective) I Intention to treatInconsistency (Heterogeneity) Independent test not part of Gold Standard

 Randomization:  Each patient has an equal (50%) chance to be in control or intervention group (Best computer generated randomization list).  Also expressed as: Sequence generated

 Concealed Allocation (assignment):  No one from research team or patients know where next patient will be allocated.  Best is remote central call. ▪ Other ways: ▪ Serially numbered opaque sealed envelope ▪ computerized with protected folder.  Also expressed as: Random allocation / Assignment

 Proper randomization and concealment prevent SELECTION BIAS  Attrition Bias

 Blinding:  Research team and patient are unaware whether patients are receiving the intervention therapy or control therapy.

 PLEASE: Do Not confuse Blinding with Allocation concealment

 Five groups should be blind: 1. Patients 2. Clinicians 3. Data collectors 4. Outcome assessors 5. Data analyst

 And the 3 Cs : 1. Co-intervention: Any extra intervention other than study treatment (either to control or intervention). 2. Contamination: Any member from one group who received intervention from other group. 3. Compliance: Adherence of participants to the intervention.

 Effective blinding and the 3 Cs (Co- intervention, Contamination and Compliance ) prevent PERFORMANCE BIAS

 Intention to treat principle:  The calculation of the outcome according to the initial involved number of study’s subjects ( i.e. : including non-adherent to treatment and drop out).  Once randomized should be analyzed ▪ These patients should e followed up for outcome occurrence.

 Another weak way Called: Per protocol analysis : Calculation of outcome after excluding the drop out.  The drop out should be < 20 % to maintain power of the study.

 Composite outcome: multiple outcomes combined together, the earliest one appears, will stop looking for other outcomes for that particular patient.  It is easier magnify the result, increase number of events, but not reflecting the real situation.

 Post hoc test: means further testing after the experiment has concluded for patterns not specified before.  The more statistical tests are done post hoc, the more chance researchers will find effective results.

 Types of bias in therapy: 1. Selection bias : if randomization or allocation was not proper. 2. Performance bias: no blinding, there is contamination or co intervention. 3. Attrition (drop out): lost follow up. 4. Measurement (detection) bias: outcome assessor not blind to risk factor when measuring outcome.

RESULTS TherapySystematic Review DiagnosisPrognosis Magnitude and Precision ARR & NNT RR,RRR and OR Pooled results: Forest Plot (RR, OR) Accuracy of the test: Sn & Sp PPV,NPV LR SnOUT SpPIN Kaplan Meier curve: -1 st year survival rate -5 yr survival rate -Median survival

APPLICAILITY TherapySystematic Review DiagnosisPrognosis Can the results be applied to my patients care? IPPP -Intervention -Patient-similar -Preference -Potential harm IPPPO -Intervention -Patient-similar -Preference -Potential harm -All outcomes considered TPPP -Test -Patient similar -Preference -Care for Patient PF -Patient similar -Follow up sufficient

2. Validity  Representative sample (criteria) ▪ PICOTT ▪ P: population ▪ I: Intervention ▪ C: Comparison ▪ O: Outcome ▪ T: Type & Quality of included study ▪ T: Time of F/U (of Study) ▪ Inclusion and exclusion

 Reproducibility (The way of process described in a sufficient way to allow reproducibility)  Article (Search / Selection / Appraisal)  Blind (quality assessment of included studies)  Inconsistency (Heterogeneity)

1) Electronic database (pubmed, medline, chocrane, embase) 2) No language restriction 3) Reference of reference 4) Conference proceedings 5) Grey literature and unpublished articles 6) Contact drug company / expert 7) Hand search on none cited articles

 At least 2 independent reviewers for quality assessment of all studies included  Can be expressed as: ▪ Risk of bias ▪ Critically evaluated ▪ We assessed / we evaluated  Can be done by using Cochrane criteria (ROB) or JADAD score

 Disagreement should be resolved (inter- observer agreement) by: 1. Third party 2. Consensus 3. Kappa test

 Interpretation of Kappa Kappa %Agreement < 40Poor 41 - 60Moderate 61 – 80Substantial > 80Almost perfect

 To compare between the new test and (index) && gold standard (Reference).

 Validity:  R ▪ Reference standard “gold” ; (acceptable, validated and practical) ▪ Representative – spectrum (sample); all levels of disease, care ;etc. ▪ Inclusion & exclusion criteria

▪ Replication (to be conducted by other people in different setting) ▪ Reproducible (is the degree to which repeated measurements under unchanged conditions show same results): ▪ Preparation of the patients ▪ Performance of the test ▪ Analysis and interpretation ▪ Precision and correctness

 A ▪ Ascertained “verification” ; I have to verify the result of the new test by using the gold standard, (Ideally, all patients should have both the index test and the gold standard)  B ▪ Blind ; when interpreting the results, investigators should not know the results of the gold standard or the index test.

 I ▪ Independent; the new test is not part of the gold standard.

 Types of bias in diagnosis:  Selection bias : did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice. (choosing only severe cases will affect the sensitivity … falsely increased)

 Verification bias: did the results of the test being evaluated influence the decision to perform the reference standard. ▪ i.e. : not all index test results will be verified by gold standard test. ▪ Solution follow up participants who were not subjected the gold standard and the duration should be based on the nature of the disease.

 Validity  R ▪ Representative sample ▪ Setting : 1˚, 2 ˚ or 3 ˚ care, if it is more from 3 ˚ car setting (more severe cases (Referred); so we will have (Referral filter bias)). ▪ Zero point (homogenous sample): all patients should be started at the same point of the clinical course/ natural history of the disease (consistent clinical course… i.e. same stage).

 A ▪ Adjustment of prognostic factor / confounders by: ▪ Matching / risk stratification  Subgrouping based on important factors  Age, gender,… ▪ Regression analysis (multi-variables)

 Prognostic factors: factor that predict which patients do better or worse (predict the outcome). ▪ i.e. association not causation  Confounder factor: independent factor not intended to be studied and will affect the result.

▪ Attrition (follow up complete and long) ▪ Duration of study enough or not (MI ≥ 3years, Cancer ≥ 5 years ) ▪ Drop out and how large  > 20 % drop out generally can affect study  How large (depend on the percentage of drop out compared to percentage of outcome)

Drop outOutcome Worst case scenario Study 11%10% 11% (minimal effect) Study21% 2% (Doubled) Example If the outcome is low and drop out is same or higher the affect will be higher (study 2)and vice versa

 B ▪ Blinding ▪ Outcome assessor should be blind to the prognostic factors ▪ Outcome clearly defined

Outcome criteria: ▪ Objective ▪ Reproducible ▪ Accurate ▪ Not composite (not combined)

 Types of bias in prognosis:  Attrition bias: (% lost to follow up or duration was not proper).  Referral filter bias: (severe cases were referred – filtered- to tertiary care because the need for special expertise).

 Diagnostic bias: outcome assessor not blind to the prognostic factors or inconsistent measurement (no clearly defined outcome criteria)  Lead time bias : survival (as measured from the time of diagnosis ) may be increase not because patients live longer but because screening lengthens the time that they have disease. E.g. prostate cancer

Study types:  Cohort (the best)  Case control

 CRQ made easy,NGH  Cebm.net

Anwar Showail.  Critical appraisal is the systematic evaluation of clinical research papers in order to establish: 1. Does this study address a clearly.

Similar presentations

Presentation on theme: "Anwar Showail.  Critical appraisal is the systematic evaluation of clinical research papers in order to establish: 1. Does this study address a clearly."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Anwar Showail.  Critical appraisal is the systematic evaluation of clinical research papers in order to establish: 1. Does this study address a clearly.

Similar presentations

Presentation on theme: "Anwar Showail.  Critical appraisal is the systematic evaluation of clinical research papers in order to establish: 1. Does this study address a clearly."— Presentation transcript:

Similar presentations

About project

Feedback