1 Bandit Thinkhamrop, PhD.(Statistics) Dept. of Biostatistics & Demography Faculty of Public Health Khon Kaen University Critical Appraisal For Health Professional

2 Core Questions 1. What is the research question? 2. What is the answer or the conclusion? 3. What is the magnitude of effect being used as the basis of the conclusion? 4. How large is the effect – conclusive or not? 5. How likely is the validity of the magnitude of effect - accurate, over or under estimate, questionable, unassessable? 6. Can the conclusion be acceptable? Why? 7. How this paper could be improved?

3 1. What is the research question? State the research question Focus on the primary research question or the one that lead to the main conclusion Example: –What are predictors of low birth weight? –What is the efficacy of the HIV vaccine?

4 2. What is the answer or the conclusion? Extract the answer to the primary research question Usually be obtained from the main conclusion of the study Example: –Low BMI of mother and not received ANC increase risk of low birth weight –HIV vaccine can prevent HIV infection

5 3. What is the magnitude of effect being used as the basis of the conclusion? If reported, transcribe it. If not, calculate it based on the reported results. Its the 95%CI is essential Summarize the magnitude of effect as a forest plot-like diagram against the meaningful level is recommended Example: –OR (95%CI) of each factor on low birth weight : Low BMI of mother (OR: 3.2; 2.50 to 4.50), and not received ANC increase risk of low birth weight (OR: 1.6; 1.02 to 2.18) –HIV vaccine efficacy is 30% (95%CI: 1.1% to 52.2%)

6 Forest plot-like diagram for the Difference   Male (0.9; 0.1 to 1.7) Female (1.4; -0.1 to 2.9) Flavor Intervention B Flavor Intervention A Overall (1.8; 0.6 to 3.0) Minimum meaningful level

7 Forest plot-like diagram for OR 0  Minimum meaningful level Low BMI of mother (3.20; 2.50 to 4.50) Protective effectRisk effect Received ANC (1.60; 1.02 to 2.18)

8 Notice the wrong scale

9

10 4. How large is the effect – conclusive or not? Compare the magnitude of both lower and upper boundary of the 95%CI with the meaningful level of effect To be conclusive, the CI needs NOT include the meaningful cut point Conclusive findings imply sufficient sample size

11 5. How likely is the validity of the magnitude of effect- accurate, over or under estimate, questionable, unassessable? Describe briefly how the magnitude of effect was obtained Describe how it could be wrong by considering these three issues: 1.Wrong data? Selection bias Information bias Confounding bias 2.Wrong analysis? Applied inappropriate statistical method 3.Poor or unclear reporting Important information was not provided

12 6. Can the conclusion be acceptable? Answer : Yes/No and Why? Explain the mechanisms as to how the magnitude of effect could be wrong, if not accepted Elaborate more by describing what position of this evidence in your evidence- base practice

13 7. How this paper could be improved? No research is perfect – there are rooms to improve. Go beyond what had been reported in the paper to what should have been done to be better under unlimited resources

14 Suggested length of the presentation (15-20 minutes) 1. Research question: 1 sentence 2. Answer: 1-2 sentences 3. Magnitude of effect: A forest plot 4. Conclusive?: Yes/No and Why? 5. Validity of the effect: –A diagram of the study design –A paragraph for brief descriptions of how the magnitude of effect was obtained –Judge if it is accurate, over or under estimate, questionable, or unassessable 6. Acceptable?: Yes/No and Why? 7. Ways to improve, if any: As needed