Presentation is loading. Please wait.

Presentation is loading. Please wait.

V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y The Scientific Method and Statistics Critical Appraisal Skills depend upon an understanding.

Similar presentations


Presentation on theme: "V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y The Scientific Method and Statistics Critical Appraisal Skills depend upon an understanding."— Presentation transcript:

1 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y The Scientific Method and Statistics Critical Appraisal Skills depend upon an understanding of the scientific method and the role statistics plays Al Best, PhD Perkinson 3100B ALBest@VCU.edu

2 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Critical Appraisal Skills Are the results of the study valid? Are the results of the study valid? What are the results? What are the results? Will the results help locally? Will the results help locally?

3 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y My goals for you Be able to answer four questions: Based on the study design, what is the level of evidence? Based on the study design, what is the level of evidence? How were threats to validity addressed? How were threats to validity addressed? Based on the goals of the study, How do you describe the results? Based on the goals of the study, How do you describe the results? To justify the conclusions, were comparisons done appropriately? To justify the conclusions, were comparisons done appropriately?

4 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Stats 2: Outline Science: Using data to answer questions Science: Using data to answer questions Case-control study (example) Case-control study (example) –Estimate prevalence in a population  Sampling, measurement, randomness  Estimates and Confidence Intervals –Compare cases and controls  Hypothesis testing  P-value, confidence interval –Interpret the results  Prevalence difference

5 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y A Scientist’s Quandary Are the results of the study valid? Are the results of the study valid? –Most experiments are highly local but have general aspirations. –How can findings generalize to other people, in other settings, with comparable interventions, and other outcomes. How will you assess whether the paper’s findings will generalize to your situation? How will you assess whether the paper’s findings will generalize to your situation? –This is the question of external validity.

6 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Solution? Use a process where sample data does generalize to the units, treatments, variables and settings not directly observed. Use a process where sample data does generalize to the units, treatments, variables and settings not directly observed. Follow the process called Statistical Inference using the Scientific Method. Follow the process called Statistical Inference using the Scientific Method.

7 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Statistical Inference

8 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Example: Barasch’s “Risk Factors for Osteonecrosis of the Jaws” “We conducted a case-control study in dental practices to determine the risk associated with bisphosphonates and to identify other risk factors for ONJ,…” “We conducted a case-control study in dental practices to determine the risk associated with bisphosphonates and to identify other risk factors for ONJ,…” From the introduction of Barasch, et al. (2011) J Dent Res 90(4), 439- 444. pubmed/21317246 pubmed/21317246

9 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Classic Steps: Inference using Statistics 1. What’s the question? (Introduction) –Conceptualize the population –State the question 2. How will you answer the question? (Methods) –The sample –The measurements –Analysis technique 4. What does it mean? (Discussion) 3. Answer the question (Results) –The sample –The measurements –Analysis technique

10 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y “a case… study … of the risk associated with BP and ONJ” Conceptualize the population Conceptualize the population –BP or ONJ? State the question: Which? State the question: Which? –In ONJ cases, estimate the prevalence of BP use. Or –In BP cases, estimate the prevalence of ONJ. Abbreviations: BP=bisphophonates, ONJ=osteonecrosis of the jaw

11 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y “a case… study … of the risk associated with BP and ONJ” Main question: Compare Main question: Compare –the prevalence of BP use higher in ONJ cases to –the prevalence of BP use higher in controls. Design a measurement system Design a measurement system –ONJ = case or control –BP = use or non-use

12 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y “a case… study … of the risk associated with BP and ONJ” Design a measurement system Design a measurement system –ONJ case

13 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Segue: Data Data is information in context. Data is information in context. That is, data is the set of information—usually numbers—arising out of measurements of individuals. That is, data is the set of information—usually numbers—arising out of measurements of individuals. Part of the information in data is also its context— how did this information come about? Part of the information in data is also its context— how did this information come about? Tonight we’re going to let the statistics speak for themselves Ed Koren, © The New Yorker, 9 December 1974.

14 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Segue: Data classification Type of data Distinguishing CharacteristicsExamples Categorical or qualitative Observations grouped into distinct classes NominalClasses without a natural order or rank Sex, treatment group, presence or absence OrdinalClasses with a predetermined or natural order Disease severity, bone density, plaque accumulation, bleeding

15 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Segue: Data classification Type of data Distinguishing CharacteristicsExamples Continuous or quantitative (numeric) Observations may assume any value on a continuous scale IntervalNumeric value with equal unit differences Temperature, GPA, age, duration of disease, income amount, serum cholesterol, number of decayed teeth Synonyms: continuous, interval

16 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y “a case… study … of the risk associated with BP and ONJ” Design a measurement system Design a measurement system –ONJ case What type of data is this? What type of data is this? –Nominal or Ordinal or Interval (continuous) ?

17 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y “a case… study … of the risk associated with BP and ONJ” Measure the subjects Measure the subjects –BP use Section H. Medications Now I would like to ask you about some of the medications that you have taken during your lifetime. This is the last section of the interview. It would be helpful to use the sheets that we sent you in our last letter. 1. Have you EVER taken any of the following drugs orally or BY MOUTH?

18 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Looking Ahead

19 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y “a case… study … of the risk associated with BP and ONJ” Measure the subjects Measure the subjects –BP use What type of data is this? What type of data is this? –Nominal or Ordinal or Interval?

20 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Backing up: State the question In ONJ cases, estimate the prevalence of BP use. In ONJ cases, estimate the prevalence of BP use. The population has a parameter—call it π —we are trying to estimate this using data. The population has a parameter—call it π —we are trying to estimate this using data.

21 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y The population has a parameter— call it π Conceptualization: The population Conceptualization: The population True:n ONJ = true count of everyone who has ONJ True:n ONJ = true count of everyone who has ONJ True:n BP = true count of everyone who has ONJ and also used BP True:n BP = true count of everyone who has ONJ and also used BP π = true prevalence proportion of BP in ONJ patients. π = true prevalence proportion of BP in ONJ patients. π = True:n BP / True:n ONJ

22 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Estimation of population parameter using sample statistic Definition: A statistic is a single descriptive number computed from the data. Definition: A statistic is a single descriptive number computed from the data. Conceptualization: The sample n ONJ = count in sample who have ONJ n ONJ = count in sample who have ONJ n BP = count in sample who have ONJ and also used BP n BP = count in sample who have ONJ and also used BP p = estimated prevalence proportion of BP in ONJ patients. p = estimated prevalence proportion of BP in ONJ patients. p = n BP / n ONJ

23 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Backing up: State the question In ONJ cases, estimate the prevalence of BP use. In ONJ cases, estimate the prevalence of BP use. π = True:n BP / True:n ONJ

24 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Classic Steps What’s the question? (Introduction) What’s the question? (Introduction) –Conceptualize the population –State the question How will you answer the question? (Methods) How will you answer the question? (Methods) –The sample –The measurements –Analysis technique What does it mean? (Discussion) What does it mean? (Discussion) Answer the question (Results) Answer the question (Results) –The sample –The measurements –Analysis technique

25 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Actual n ONJ ? Actual n BP ? Analyze the data Analyze the data –BP use in ONJ patients –n ONJ = count in sample who have ONJ –n BP = count in sample who have ONJ and also used BP p = estimated prevalence proportion of BP in ONJ patients. p = estimated prevalence proportion of BP in ONJ patients. p = n BP / n ONJ

26 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Estimate of p? Analyze the data Analyze the data –BP use –From sample nONJ=191, in these nBP=113 p = 113/191 ?= 83% = proportion 0.83 –Point estimate of p = 0.83

27 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Segue: estimation error In ONJ cases, estimate the prevalence of BP use. In ONJ cases, estimate the prevalence of BP use. Variability due to sampling: 308 cases down to 191 Variability due to measurement: n BP = 113 n NoBP = 24 Unknown = 24.825=113/137

28 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Answer the question In ONJ cases, estimate the prevalence of BP use. In ONJ cases, estimate the prevalence of BP use. The population has a parameter—call it π —we are trying to estimate—using data. The population has a parameter—call it π —we are trying to estimate—using data. In ONJ patients, 83% reported BP use

29 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Classic Steps What’s the question? (Introduction) What’s the question? (Introduction) –Conceptualize the population –State the question How will you answer the question? (Methods) How will you answer the question? (Methods) –The sample –The measurements –Analysis technique What does it mean? (Discussion) What does it mean? (Discussion) Answer the question (Results) Answer the question (Results) –The sample –The measurements –Analysis technique

30 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Defining the research question: Testable consequence? Conceptual progression from general to specific Conceptual progression from general to specific General question General question –Is Bisphosphonate use a risk factor for ONJ? Specific hypothesis Specific hypothesis –Is the prevalence of bisphosphonate use higher in ONJ cases than in controls? Testable consequence Testable consequence –Prediction of a relationship –Potentially refutable by data

31 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Defining the research question: Prediction Prediction: a statistical relationship between exposure and outcome Prediction: a statistical relationship between exposure and outcome –BP prevalence will be higher in ONJ cases than in controls How do we arrive at this? Using a refutable hypothesis How do we arrive at this? Using a refutable hypothesis

32 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Defining the research question: Formalization A refutable hypothesis A refutable hypothesis Statistical formalization: Statistical formalization: Ho: proportion BP (ONJ) = proportion BP (controls) Ho: proportion BP (ONJ) = proportion BP (controls) –Which may be disproved beyond a reasonable doubt through falsification by data via statistical hypothesis testing, in favor of: Ha: proportion BP (ONJ) > proportion BP (controls) Ha: proportion BP (ONJ) > proportion BP (controls)

33 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Critical Appraisal Is it a testable, research question? Is it a testable, research question? How did they try to rule out bias, confounding, chance? How did they try to rule out bias, confounding, chance? How did they consider multiple outcome measures and multiple predictors? How did they consider multiple outcome measures and multiple predictors? Did they disclose what was done with enough detail so others may replicate? Did they disclose what was done with enough detail so others may replicate?

34 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y How Science Advances Clinical Knowledge Science forms a question and brings data to bear to answer the question. Science forms a question and brings data to bear to answer the question. Informally: Informally: 1.Frame a clinical research question. 2.State its testable consequences as either “just random variability” or “unusual outcomes”. 3.Compare the actual data with these two choices and decide which to believe. 4.Discuss our present understanding.

35 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Testing Hypotheses Or, linking the four steps to the standard IMRD organization of a paper: Or, linking the four steps to the standard IMRD organization of a paper: 1.What’s the question? (Introduction) 2.How do you answer the question? (Methods) 3.Answer the question. (Results) 4.What does it mean? (Discussion)

36 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y What’s the Question? “We conducted a case-control study in dental practices to determine the risk associated with bisphosphonates and to identify other risk factors for ONJ,…” “We conducted a case-control study in dental practices to determine the risk associated with bisphosphonates and to identify other risk factors for ONJ,…” From the introduction of Barasch, et al. (2011) J Dent Res 90(4), 439-444.

37 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y How do you answer the question? Outline Outline –Propose two states of nature –Use the rule of simplicity –Take into account that “noise happens” –Use a test statistic to decide: Signal or Noise?

38 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Two states; Two hypotheses We begin by conceiving the true state of nature as being either: We begin by conceiving the true state of nature as being either: –no difference or –a difference. We always start by assuming that nothing is going on—that any apparent differences are purely because of chance. Our preference, as scientists, is to believe the simplest explanation for a phenomenon. We always start by assuming that nothing is going on—that any apparent differences are purely because of chance. Our preference, as scientists, is to believe the simplest explanation for a phenomenon. –Assume: no difference (AKA null hypothesis).

39 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Rule of Simplicity When you have two competing theories which make exactly the same predictions, the one that is simpler is the better. When you have two competing theories which make exactly the same predictions, the one that is simpler is the better. The simplest explanation for some phenomenon is more likely to be accurate than more complicated explanations. The simplest explanation for some phenomenon is more likely to be accurate than more complicated explanations. The explanation requiring the fewest assumptions is most likely to be correct. The explanation requiring the fewest assumptions is most likely to be correct. AKA “Occam’s Razor”

40 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Statistical World View Thou shalt not interpret randomness. Thou shalt not interpret randomness. Chance happens. Noise exists. Making an interpretation that goes beyond this requires justification. Chance happens. Noise exists. Making an interpretation that goes beyond this requires justification. If random noise, measurement error, or chance occurrence can account for variations (differences) in the observations, then there is no need to formulate a more complicated explanation. If random noise, measurement error, or chance occurrence can account for variations (differences) in the observations, then there is no need to formulate a more complicated explanation. –We embody this preference in the statement of the null hypothesis.

41 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Null Hypothesis We evaluate this proposition using statistical techniques. We evaluate this proposition using statistical techniques. The null hypothesis is the statement that is tested. It’s abbreviated H0: The null hypothesis is the statement that is tested. It’s abbreviated H0: A null-hypothesis is the simplest explanation of events: There is no difference. There is no change. There is no improvement. Nothing unusual is occurring. A null-hypothesis is the simplest explanation of events: There is no difference. There is no change. There is no improvement. Nothing unusual is occurring. A null-hypothesis is the statement we hope to contradict with data. That is, we usually hope to reject the null hypothesis. A null-hypothesis is the statement we hope to contradict with data. That is, we usually hope to reject the null hypothesis.

42 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Assume: Nothing is going on Prevalence of bisphosphonate use within those who do have ONJ is equal to the prevalence of bisphosphonate use within those who do not have ONJ. Prevalence of bisphosphonate use within those who do have ONJ is equal to the prevalence of bisphosphonate use within those who do not have ONJ. HO: π cases = π controls HO: π cases = π controls

43 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Two Hypotheses Prevalence of bisphosphonate use within those who have ONJ is equal to the prevalence of bisphosphonate use within those who do not have ONJ. Prevalence of bisphosphonate use within those who have ONJ is equal to the prevalence of bisphosphonate use within those who do not have ONJ. HO: Pcases = Pcontrols HO: Pcases = Pcontrols Can we reject the above, in favor of: Prevalence of bisphosphonate use within those who have ONJ is different than the prevalence of bisphosphonate use within those who do not have ONJ. Prevalence of bisphosphonate use within those who have ONJ is different than the prevalence of bisphosphonate use within those who do not have ONJ. HA: Pcases ≠ Pcontrols HA: Pcases ≠ Pcontrols So: Done with step 1: We’ve stated the question. Next: How will you answer the question?

44 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Proof By Contradiction Nature is either in one state or the other. Nature is either in one state or the other. –We prefer to believe the simplest explanation. Collect data from the real world. Collect data from the real world. Assess the likelihood of observing this data under the null hypothesis. Assess the likelihood of observing this data under the null hypothesis. Choose to believe: Choose to believe: –If the data is within what we would expect then we retain our preference for the null-hypothesis as the best explanation. –If the data is very unlikely, then we reject the null hypothesis in favor of its alternative.

45 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y What if: HO is true? In ONJ cases, estimate the prevalence of BP use. In ONJ cases, estimate the prevalence of BP use. In controls, estimate the prevalence of BP use. In controls, estimate the prevalence of BP use. π = π ONJ = π control

46 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y What if? Assess the likelihood of observing various data possibilities under the null hypothesis. Assess the likelihood of observing various data possibilities under the null hypothesis. Assume this is true: Assume this is true: –HO: P cases = P controls Then the sample estimate of P cases will be “close to” the sample estimate of P controls. Then the sample estimate of P cases will be “close to” the sample estimate of P controls. –By “close to” we mean that, because of sampling variability and measurement error we expect them to be somewhat different.

47 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Contingency Table “Results: … Therefore, 191 cases together with 573 controls were included in the analyses….Bisphosphonate use was reported by 113 cases (83%) and 71 controls (15%),…” Note: 113+71=184. 473 137

48 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Contingency Table Assume this is true: Assume this is true: –HO: Pcases = Pcontrols = 0.302

49 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Contingency Table Assume this is true: Assume this is true: –HO: Pcases = Pcontrols = 0.302 –30.2% of 137= 41

50 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Contingency Table Assume this is true: Assume this is true: –HO: Pcases = Pcontrols = 0.302 –30.2% of 473 = 143

51 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Contingency Table Assume this is true: Assume this is true: –HO: Pcases = Pcontrols = 0.302 –137 – 41 = 96 –473 – 143 = 330

52 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Test Statistic Test stat: Difference in prevalence Test stat: Difference in prevalence –HO: Pcases – Pcontrols = 0.0 –Expected difference = 0%

53 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Testing Hypotheses Recall: Recall: –Frame a clinical research question. –State its testable consequences as either “just random variability” or “unusual outcomes”. –Compare the actual data with these two choices and decide which to believe. –Discuss our present understanding.

54 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y I presume the null-hypothesis is true, do the data support this? Observed difference = 0% Observed difference = 0% P-value = 1.0 P-value = 1.0

55 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y I presume the null-hypothesis is true, do the data support this? Observed difference = 0.6% Observed difference = 0.6% P-value = 0.885 P-value = 0.885

56 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y I presume the null-hypothesis is true, do the data support this? Observed difference = 1.6% Observed difference = 1.6% P-value = 0.724 P-value = 0.724

57 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y I presume the null-hypothesis is true, do the data support this? Observed difference = 90% Observed difference = 90% P-value < 0.001 P-value < 0.001

58 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Trade offs Alpha = Type I error = prob. of rejecting a true null hypothesis Beta = Type II error = prob. of not finding a true difference Conclusion Truth Do not reject null-hypothesis (p-value >.05) Reject null- hypothesis (p-value <.05) Null-hypothesis (no difference) correctType I error Alternative hypothesis (difference) Type II errorcorrect

59 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Significance Level The significance level is represented by the Greek symbol “alpha”, α. It is the probability of rejecting a true null hypothesis. The researcher chooses the risk of making this error: concluding that the null hypothesis is false when it really is true. – –The most typical values are α =.05,.01, or.10.

60 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Universal Decision Rule Choose to believe: Choose to believe: HO: null-hypothesis HO: null-hypothesis (For non-extreme values of the test statistic) –Choose this if p-value ≥ (usually 0.05) –Choose this if p-value ≥ α (usually 0.05) HA: alternative-hypothesis HA: alternative-hypothesis (For extreme values of the test statistic) –Choose this if p-value <(usually 0.05) –Choose this if p-value < α (usually 0.05)

61 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y ! Are we done yet? ! What’s the question? (Introduction) What’s the question? (Introduction) –Conceptualize the population –State the question How will you answer the question? (Methods) How will you answer the question? (Methods) –The sample –The measurements –Analysis technique What does it mean? (Discussion) What does it mean? (Discussion) Answer the question (Results) Answer the question (Results) –The sample –The measurements –Analysis technique

62 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y I presume the null-hypothesis is true, do the data support this? Observed difference = 67.5% Observed difference = 67.5% P-value <.0001 P-value <.0001

63 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Chi-square test Expected data Expected data Observed data (chi-square, P-value <.0001) Observed data (chi-square, P-value <.0001)

64 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y State a Conclusion Prevalence of bisphosphonate use within those who have ONJ is equal to the prevalence of bisphosphonate use within those who do not have ONJ. Prevalence of bisphosphonate use within those who have ONJ is equal to the prevalence of bisphosphonate use within those who do not have ONJ. –HO: Pcases = Pcontrols Prevalence of bisphosphonate use within those who have ONJ is different than the prevalence of bisphosphonate use within those who do not have ONJ. Prevalence of bisphosphonate use within those who have ONJ is different than the prevalence of bisphosphonate use within those who do not have ONJ. –HA: Pcases ≠ Pcontrols Evidence: 82.5% prevalence vs 15%, p-value <.0001 Evidence: 82.5% prevalence vs 15%, p-value <.0001

65 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y P-value The p-value is the probability that the data occurred by chance, assuming the null hypothesis is true. The p-value is the probability that the data occurred by chance, assuming the null hypothesis is true. The p-value is NOT the probability that the null- hypothesis is true. The p-value is NOT the probability that the null- hypothesis is true.

66 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y The p-value is NOT the probability that the null-hypothesis is true. and: 1−pvalue is NOT the probability that the alternative hypothesis is true

67 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Trade offs Alpha = Type I error = prob. of rejecting a true null hypothesis Beta = Type II error = prob. of not finding a true difference Conclusion Truth Do not reject null-hypothesis (p-value >.05) Reject null- hypothesis (p-value <.05) Null-hypothesis (no difference) correctType I error Alternative hypothesis (difference) Type II errorcorrect

68 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Actuality Alpha = Type I error = prob. of rejecting a true null hypothesis Beta = Type II error = prob. of not finding a true difference Results Truth Do not reject null-hypothesis (p-value >.05) Reject null- hypothesis (p-value <.05) Null-hypothesis (no difference) Blind alley ? Alternative hypothesis (difference)?Discovery!

69 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y P-value A modest reality: A modest reality: The p-value is simply the probability that the data occurred by chance. Big leap: Big leap: A significant p-value is a license to make up a story.

70 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Results The Prevalence of bisphosphonate use within those who have ONJ is different than the prevalence of bisphosphonate use within those who do not have ONJ (p-value <.0001). The Prevalence of bisphosphonate use within those who have ONJ is different than the prevalence of bisphosphonate use within those who do not have ONJ (p-value <.0001).Discussion “In conclusion, this case-control study supports a causal link between bisphosphonates and ONJ”* * See page 443, the last Discussion paragraph. “In conclusion, this case-control study supports a causal link between bisphosphonates and ONJ”* * See page 443, the last Discussion paragraph.

71 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Review To assess validity To assess validity –In the study, What’s the question? –Where did the data come from? Sampling and measurement. –What would you expect if “nothing is going on”? –Is the observed data different than that? But other factors could account for the observed difference But other factors could account for the observed difference –Bias, confounding, multiplicity

72 V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y Next Time Be able to answer four questions: Based on the study design, what is the level of evidence? Based on the study design, what is the level of evidence? How were threats to validity addressed? How were threats to validity addressed? Based on the goals of the study, How do you describe the results? Based on the goals of the study, How do you describe the results? To justify the conclusions, were comparisons done appropriately? To justify the conclusions, were comparisons done appropriately?


Download ppt "V I R G I N I A C O M M O N W E A L T H U N I V E R S I T Y The Scientific Method and Statistics Critical Appraisal Skills depend upon an understanding."

Similar presentations


Ads by Google