Statistical and Practical Significance Advanced Statistics Petr Soukup.

Slides:

Advertisements

Similar presentations

Chapter 16 Inferential Statistics

Advertisements

Hypothesis Testing making decisions using sample data.

Inferential Statistics

SAMPLE DESIGN: HOW MANY WILL BE IN THE SAMPLE—ANALYTICAL OR EXPERIMENTAL STUDIES ? Lu Ann Aday, Ph.D. The University of Texas School of Public Health.

Chapter Seventeen HYPOTHESIS TESTING

10-1 Introduction 10-2 Inference for a Difference in Means of Two Normal Distributions, Variances Known Figure 10-1 Two independent populations.

Dr. Michael R. Hyman, NMSU Differences Between Group Means.

Hypothesis Testing Steps of a Statistical Significance Test. 1. Assumptions Type of data, form of population, method of sampling, sample size.

Sample size computations Petter Mostad

9-1 Hypothesis Testing Statistical Hypotheses Statistical hypothesis testing and confidence interval estimation of parameters are the fundamental.

Stat Day 16 Observations (Topic 16 and Topic 14)

BCOR 1020 Business Statistics Lecture 22 – April 10, 2008.

BCOR 1020 Business Statistics Lecture 28 – May 1, 2008.

Chapter 9 Hypothesis Testing 9.4 Testing a Hypothesis about a Population Proportion.

4-1 Statistical Inference The field of statistical inference consists of those methods used to make decisions or draw conclusions about a population.

BCOR 1020 Business Statistics Lecture 20 – April 3, 2008.

BCOR 1020 Business Statistics

Chapter 9 Hypothesis Testing II. Chapter Outline  Introduction  Hypothesis Testing with Sample Means (Large Samples)  Hypothesis Testing with Sample.

Chapter 9 Hypothesis Testing II. Chapter Outline  Introduction  Hypothesis Testing with Sample Means (Large Samples)  Hypothesis Testing with Sample.

Simple Linear Regression Analysis

Chapter 9 Title and Outline 1 9 Tests of Hypotheses for a Single Sample 9-1 Hypothesis Testing Statistical Hypotheses Tests of Statistical.

Choosing Statistical Procedures

AM Recitation 2/10/11.

Chapter 13 – 1 Chapter 12: Testing Hypotheses Overview Research and null hypotheses One and two-tailed tests Errors Testing the difference between two.

1 © Lecture note 3 Hypothesis Testing MAKE HYPOTHESIS ©

Chapter 8 Inferences Based on a Single Sample: Tests of Hypothesis.

1/2555 สมศักดิ์ ศิวดำรงพงศ์

Comparing Two Proportions

Statistical Inference Decision Making (Hypothesis Testing) Decision Making (Hypothesis Testing) A formal method for decision making in the presence of.

Chapter 9 Hypothesis Testing II: two samples Test of significance for sample means (large samples) The difference between “statistical significance” and.

10-1 Introduction 10-2 Inference for a Difference in Means of Two Normal Distributions, Variances Known Figure 10-1 Two independent populations.

9-1 Hypothesis Testing Statistical Hypotheses Definition Statistical hypothesis testing and confidence interval estimation of parameters are.

1 10 Statistical Inference for Two Samples 10-1 Inference on the Difference in Means of Two Normal Distributions, Variances Known Hypothesis tests.

Statistical Hypotheses & Hypothesis Testing. Statistical Hypotheses There are two types of statistical hypotheses. Null Hypothesis The null hypothesis,

4 Hypothesis & Testing. CHAPTER OUTLINE 4-1 STATISTICAL INFERENCE 4-2 POINT ESTIMATION 4-3 HYPOTHESIS TESTING Statistical Hypotheses Testing.

1 9 Tests of Hypotheses for a Single Sample. © John Wiley & Sons, Inc. Applied Statistics and Probability for Engineers, by Montgomery and Runger. 9-1.

 Descriptive Methods ◦ Observation ◦ Survey Research  Experimental Methods ◦ Independent Groups Designs ◦ Repeated Measures Designs ◦ Complex Designs.

1 CHAPTER 4 CHAPTER 4 WHAT IS A CONFIDENCE INTERVAL? WHAT IS A CONFIDENCE INTERVAL? confidence interval A confidence interval estimates a population parameter.

Chapter 9: Testing Hypotheses Overview Research and null hypotheses One and two-tailed tests Type I and II Errors Testing the difference between two means.

Chapter 10 The t Test for Two Independent Samples.

Hypothesis Testing Errors. Hypothesis Testing Suppose we believe the average systolic blood pressure of healthy adults is normally distributed with mean.

T Test for Two Independent Samples. t test for two independent samples Basic Assumptions Independent samples are not paired with other observations Null.

Statistical Inference Drawing conclusions (“to infer”) about a population based upon data from a sample. Drawing conclusions (“to infer”) about a population.

Chapter 22 Comparing Two Proportions.  Comparisons between two percentages are much more common than questions about isolated percentages.  We often.

T tests comparing two means t tests comparing two means.

PEP-PMMA Training Session Statistical inference Lima, Peru Abdelkrim Araar / Jean-Yves Duclos 9-10 June 2007.

Lec. 19 – Hypothesis Testing: The Null and Types of Error.

CHAPTER 7: TESTING HYPOTHESES Leon-Guerrero and Frankfort-Nachmias, Essentials of Statistics for a Diverse Society.

McGraw-Hill/Irwin © 2003 The McGraw-Hill Companies, Inc.,All Rights Reserved. Part Four ANALYSIS AND PRESENTATION OF DATA.

1 Basics of Inferential Statistics Mark A. Weaver, PhD Family Health International Office of AIDS Research, NIH ICSSC, FHI Lucknow, India, March 2010.

Part Four ANALYSIS AND PRESENTATION OF DATA

STAT 312 Chapter 7 - Statistical Intervals Based on a Single Sample

STA 291 Spring 2010 Lecture 18 Dustin Lueker.

Hypothesis Testing II: The Two-sample Case

Statistics in SPSS Lecture 7

MATH 2311 Section 8.2.

Hypothesis Theory PhD course.

Comparing Two Proportions

Hypothesis Tests for a Population Mean in Practice

P-value Approach for Test Conclusion

Chapter 6 Making Sense of Statistical Significance: Decision Errors, Effect Size and Statistical Power Part 1: Sept. 18, 2014.

9 Tests of Hypotheses for a Single Sample CHAPTER OUTLINE

Comparing Two Proportions

CHAPTER 12 Inference for Proportions

STA 291 Spring 2008 Lecture 18 Dustin Lueker.

CHAPTER 12 Inference for Proportions

Statistical and Practical Significance

STA 291 Summer 2008 Lecture 18 Dustin Lueker.

Statistical Power.

Presentation transcript:

Statistical and Practical Significance Advanced Statistics Petr Soukup

Outline Reminder of statistical significance Limits of statistical significance Misuses of statistical significance Alternatives to statistical significance Practical significance Effect sizes

REMINDER OF STATISTICAL SIGNIFICANCE (NHST)

Hypotheses and tests Tested hypothesis in experiments (Fisher, 1925) Null and alternative hypothesis (NHST) (Neyman&Pearson, 1937) Common tests - t-tests, analysis of variance, analysis of covariance, correlation analysis etc.

Definition of statistical significance Decision True statusH0H1 H0OK (P=1- α)Type I error (P= α) H1Type II error (P= β)OK (P= 1-β) Test Power Definition: Conditional probability, that our sample can be drawn from population in which null hypothesis is valid (α). Statistical significance is P(D/H0) and not P(H0/D)

LIMITS OF NHST

Assumptions for classical NHST Big big probability samples from infinite or very big finite populations Three assumptions: Big (infinite) population (at least 100times bigger than the sample) Probability sampling (all units same probability of selection) Big sample (> units)

LIMITS OF NHST 1. data from censuses 2. data from non-probability samples 3. data from small samples 4. data based on sample that are big proportion of the basic population 5. big data samples from merged (internationally or by time) files

Beyond the limits of NHST in CSR* *CSR-Czech sociological review N=32 articles, Czech sociological review (selected 29 issues), own research

MISUSES OF NHST

Objections against NHST (Misuses of NHST) a) Insufficient statement about population, b) null hypotheses are unreal (nill null), c) mechanical usage of classical 5% statistical significance (asterisks, stepwise methods, best models etc.), d) statistical significant doesn’t mean important, e) publishing only statistical significant results (file drawer problem).

Misuses of NHST in CSR* *CSR-Czech sociological review N=32 articles, Czech sociological review (selected 29 issues), own research

Conference examples (P<0,01)

Conference examples (***)

Conference examples (*** and stepwise)

ALTERNATIVES TO NHST

Some alternatives to statistical significance a) Confidence Intervals (Problems for r, formulas, regression etc.) b) Test power (quite good in sociology), c) Estimate of minimum sample size & What if strategy, d) Comparison of models via information criterias (AIC, BIC) e) Bayesian approach

PRACTICAL SIGNIFICANCE

Practical significance - terminology a) a)Practical significance b) Substantive significance c) Logical significance d) Scientific significance sometimes also: e) result importance or f) result meaningfulness

How to measure Practical sig.? History - Absolute and relative approach Example: Income differencies Absolute and relative difference

How to measure Practical sig.? Effect sizes – measures of practical significance Some well known: Cohen d Hayes ω But also R 2, r, C, Fisher η 2 are effect sizes Problem: Sometimes published but not interpreted

OTHER SIGNIFICANCES

Special significances Economic significance Clinical significance Etc.

CONCLUSION? Statistical significance is: LIMITED MISUSED BUT NOT BAD Substantive significance is: NOT OFTEN USED BUT NECESSARY