Associate Professor Arthur Dryver, PhD School of Business Administration, NIDA url:

Know your respondents

Taking a sample from the population Population Example: 10 million customers Example: A sample of 80 respondents

Are the sample results important? The sample is important in what it says about the population. A company with 10 million customers will survive if 80 clients are unhappy and ultimately leave due to disappointment in the product. A company will not survive if those 80 unhappy customers represent the 10 million customers and most 10 million are considering leaving. How can we tell?

Do you have enough data? Gender breakout in a sample survey

Do you have enough data? Age breakout in a sample survey Don’t make conclusions based on only a few respondents. Different ages probably different concerns

Do you have enough data to answer your questions?

Does your data represent your population? Gender breakout in a sample survey Different ages possibly different concerns. If so this survey can be misleading if your population consists of 50% each. The responses biased toward females.

Does your data represent your population? Age Breakout in a sample survey Different ages probably different concerns

Know your respondents Does your data represent your population of interest? If about VIP customers then does it represent your typical VIP customer Also, think about the respondents What is their incentive to respond? Don’t make broad conclusions based on a misrepresentative sample. Make statements considering taking into account your data

Probability Flipping a coin – probability of heads (1/2) Probability of 2 heads out of 2 = ¼ Rolling a die with 6 sides. Probability of a 4 = 1/6 What if we don’t know anything how can we calculate probabilities? How much people spend on phone apps per month? Are they satisfied with your service? etc.

Central Limit Theorem

Central Limit Theorem (CLT) For a large sample size n>30

The importance of central limit theorem Generally we cannot calculate probabilities on a single observation because we don’t know enough. We can calculate probabilities on the sample average making some assumptions because we know its probability distribution function.

Hypothesis Testing Why???? Why not just look at the mean and percentages??? Men spend more than women. Done or are we???

Hypothesis Testing What if this came from 10 men and 10 women? What if 100 of each? How can we take what we’ve learned from the survey and discuss about our 10 million customers? This is the benefit behind hypothesis testing - going from a sample to population. Yes 300>250, yes men spend more than women in the sample. Now can we make a definite statement about the population – 15 million customers?

Taking a sample from the population Population Example: 15 million customers Example: A sample of 200 respondents How can we examine 200 and talk about 15 million????

Hypothesis testing In general within hypothesis testing we wish to test a theory, belief or simply something of interest. It is desired to test if a quantity concerning the population, called a parameter, is either not equal to, greater than or less than some value (Alternative hypothesis). Null hypothesis: =, =. Often within hypothesis testing one may want to compare two groups/samples to each other E.g. comparing the population average salary of men, to the population average salary of women.

Examples

P-Value The p-value is a probability. The null hypothesis is used to calculate the probability. The p-value is the probability of observing the test statistic or more extreme given the null hypothesis.

Steps Within Hypothesis Testing: P-value Approach 1. Determine the population of interest – Example: True customers 2. Determine the null hypothesis, and the alternative hypothesis. 3. Decide on the appropriate level of significance, alpha. 4. Determine the sample size and sampling design to use. 1. The tests in this chapter are appropriate when the data comes from a simple random sample. 2. Most statistical tests are not appropriate when the data comes from a convenience or other types of non-probability sample. What is done and what should be done are often not the same. 5. Determine the appropriate test statistic given the data and sampling design. 6. Collect the data and calculate the appropriate test statistic. 7. Calculate the p-value for the null and alternative hypothesis combination. 8. Make a decision whether to fail to reject or reject the null hypothesis by comparing the p-value to alpha.

Conclusions from a hypothesis Test When a hypothesis test is performed, the result is either fail to reject the null hypothesis or reject the null hypothesis. Do not say "accept" the null hypothesis. There is a huge difference between not having enough evidence to disprove something and proving something.

Conclusions from a hypothesis Test Null hypothesis Men spend <= Women on phone apps Alternative hypothesis Men spend > Women on phone apps Reject Null hypothesis means we are confident that men spend more than women. Fail to reject means we are not confident – need more evidence. Yes 300>250 but this is only the sample, can we be certain about the population?

Types of Error is true is false Fail to rejectP(No error)= P(Type II Error)= RejectP(Type I error)= P(No error)=

Hypothesis Tests: Formulas A larger n and everything else assuming the same leads to a larger test statistic – increases the probability of rejecting the null hypothesis.

Statistical Significance Versus Practical Significance Yes 251>250, yes men spend more than women in the sample. With enough data this will be statistically significant, but it is not practically significant. Don’t make big marketing decisions over small differences – Example: like a 1 Baht difference.

What to do when.

First off, what is the number of variables and the type of variable(s): One variable: categorical continuous Two variables: categorical and categorical categorical and continuous continuous and continuous Multiple variables: Continuous dependent Categorical dependent Exception: Time series data - use time series techniques

One Variable Categorical Two Categories Binomial Test or Z approximation for test of proportions More than two categories Chi-square test Continuous t-test

Two Variables Categorical and Categorical (nominal data both) Two Categories For Each Variable If one or two sided test: Two Sample Z-test of proportions If two sided test: Chi-Square test More than two categories for at least one of the variables. Chi-square test Categorical (nominal) and Categorical (ordinal data e.g. strongly disagree to strongly agree 1 to 5 – with a single question – not average of say 5 questions) Two categories Consider non-parametric statistical tests E.g. Mann-Whitney and Wilcoxon tests Multiple categories Consider non-parametric statistical tests

Two Variables Categorical and Continuous Two Categories Two independent sample t-test Paired t-test (if data is paired) More than two Categories ANOVA Continuous and Continuous Correlation and simple linear regression

Multiple Variables Multiple variables: Continuous dependent General linear model Multiple variables: Categorical dependent Two Categories Logistic regression More than two Categories Discriminant analysis Time Series Time series data - use time series techniques

Practice Practice Practice