Conducting A Study Designing Sample Designing Experiments Simulating Experiments Designing Sample Designing Experiments Simulating Experiments

Introduction  Observational study: observes and measure variables regarding individuals; no influence.  Experimental: imposes some treatment on individuals to observe their responses.  Observational study: observes and measure variables regarding individuals; no influence.  Experimental: imposes some treatment on individuals to observe their responses.

Introduction  Population: The collection of people, animals, or things to study  Sample: A subset of the population  Parameters: Calculations on the entire population  Statistics: Calculations on sample data set  Population: The collection of people, animals, or things to study  Sample: A subset of the population  Parameters: Calculations on the entire population  Statistics: Calculations on sample data set

Introduction  Bias: If the sample is not representative of the population, but differs substantially in the characteristic of interest, we say that the sample is biased.  Census: attempts to contact every individual in the entire population.  Bias: If the sample is not representative of the population, but differs substantially in the characteristic of interest, we say that the sample is biased.  Census: attempts to contact every individual in the entire population.

Designing Samples: Suppose we want to gather information about a group of people.  If the group is small (for example, all students in this class) we can study each group member directly.  If, however, the group is very large (for example, all students in the school), studying each member of the group may not be feasible.  The method we use to select the sample is called the sample design. The design of the sample is very important. If the design is poor, the sample will not accurately represent the population. Suppose we want to gather information about a group of people.  If the group is small (for example, all students in this class) we can study each group member directly.  If, however, the group is very large (for example, all students in the school), studying each member of the group may not be feasible.  The method we use to select the sample is called the sample design. The design of the sample is very important. If the design is poor, the sample will not accurately represent the population.

Designing Samples:  Here’s the most important overarching concept regarding obtaining samples:  Probability Sample: “a sample chosen by chance. We must know what samples are possible and what chance, or probability, each possible sample has.”  Here’s the most important overarching concept regarding obtaining samples:  Probability Sample: “a sample chosen by chance. We must know what samples are possible and what chance, or probability, each possible sample has.”

Types of Sample Designs:  Voluntary Response Sample  People choose themselves to be in the sample by responding to a general appeal  Example: We post an advertisement in Bay Eagle asking ESHS students to respond  Problem: People with strong opinions (often strong negative opinions) tend to reply, so they are overrepresented  Voluntary Response Sample  People choose themselves to be in the sample by responding to a general appeal  Example: We post an advertisement in Bay Eagle asking ESHS students to respond  Problem: People with strong opinions (often strong negative opinions) tend to reply, so they are overrepresented

Types of Sample Designs:  Convenience Sample  Individuals who are easiest to reach are chosen for the sample  Example: We use students in this class as our sample  Problem: This group may not be diverse enough to accurately represent all students at ESHS  Convenience Sample  Individuals who are easiest to reach are chosen for the sample  Example: We use students in this class as our sample  Problem: This group may not be diverse enough to accurately represent all students at ESHS

Types of Sample Designs:  Both Voluntary Response Samples and Convenience Samples result in a sample that is not representative of the population. These are biased samples because they favor certain outcomes.  Random selection eliminates bias from sample choice.  Both Voluntary Response Samples and Convenience Samples result in a sample that is not representative of the population. These are biased samples because they favor certain outcomes.  Random selection eliminates bias from sample choice.

Types of Sample Designs:  Simple Random Sample (SRS)  Individuals are selected so that all possible combinations of individuals are equally likely to be in the sample  Example: Generate a list of student ID numbers for all students at ESHS; then randomly select student ID numbers and choose those students for the sample  Simple Random Sample (SRS)  Individuals are selected so that all possible combinations of individuals are equally likely to be in the sample  Example: Generate a list of student ID numbers for all students at ESHS; then randomly select student ID numbers and choose those students for the sample

How to Select a SRS  We will actually do this in class together.

Other Types of Sample Designs:  Systematic Random Sample  The first individual is chosen at random; then a system or rule is used to choose all other individuals  Example: Obtain an alphabetized list of all students at ESHS. Choose every 5 th person on the list.  Systematic Random Sample  The first individual is chosen at random; then a system or rule is used to choose all other individuals  Example: Obtain an alphabetized list of all students at ESHS. Choose every 5 th person on the list.

Other Types of Sample Designs:  Stratified Random Sample  Divide the population into groups of similar individuals; choose a SRS in each group to form the full sample  Example: Divide all of the students at ESHS into four groups: freshmen, sophomores, juniors, and seniors; the choose a SRS from each grade level  Stratified Random Sample  Divide the population into groups of similar individuals; choose a SRS in each group to form the full sample  Example: Divide all of the students at ESHS into four groups: freshmen, sophomores, juniors, and seniors; the choose a SRS from each grade level

Other Types of Sample Designs:  Multistage Sample  Select several groups; within each group, select a subgroup; within each subgroup select individuals for the sample.  Example: Select several departments within the school (Math, English, Art). Within each of those departments, select several teachers. Choose several students within each class.  Multistage Sample  Select several groups; within each group, select a subgroup; within each subgroup select individuals for the sample.  Example: Select several departments within the school (Math, English, Art). Within each of those departments, select several teachers. Choose several students within each class.

Other Types of Sample Designs:  Cluster Sample  Select several groups; within each group, select several subgroups; within each subgroup select ALL individuals for the sample.  Example: Select several departments within the school (Math, English, Art). Within each of those departments, select several teachers. Choose ALL students in each class.  Cluster Sample  Select several groups; within each group, select several subgroups; within each subgroup select ALL individuals for the sample.  Example: Select several departments within the school (Math, English, Art). Within each of those departments, select several teachers. Choose ALL students in each class.

 Although random selection eliminates bias from our choice of sample, it does not guarantee that our sample is representative of the population.  Potential problems include:  Although random selection eliminates bias from our choice of sample, it does not guarantee that our sample is representative of the population.  Potential problems include:

Potential problems include:  Undercoverage:  Some groups are left out of the process of choosing the sample  Example: Students in SCROC, early release, on suspension, or absent may be left out of the sample  Undercoverage:  Some groups are left out of the process of choosing the sample  Example: Students in SCROC, early release, on suspension, or absent may be left out of the sample

Potential problems include:  Nonresponse:  An individual chosen for the sample cannot be contacted or refuses to cooperate  Example: A student chosen for the sample may refuse to divulge information or may be absent  Nonresponse:  An individual chosen for the sample cannot be contacted or refuses to cooperate  Example: A student chosen for the sample may refuse to divulge information or may be absent

Potential problems include:  Response Bias  The behavior of the individual or interviewer may influence the accuracy of the response  Example: Students may lie about drug or alcohol use  Response Bias  The behavior of the individual or interviewer may influence the accuracy of the response  Example: Students may lie about drug or alcohol use

Potential problems include:  Wording of Questions  Confusing or leading questions influence responses; poorly worded questions will not yield accurate responses.  Example 1: “ In a recent study, students in an Algebra I course were given a 25 question basic skills test. On average, students used a graphing calculator to answer 21 out of 25 questions. Do you think graphing calculators are overused? ”  Wording of Questions  Confusing or leading questions influence responses; poorly worded questions will not yield accurate responses.  Example 1: “ In a recent study, students in an Algebra I course were given a 25 question basic skills test. On average, students used a graphing calculator to answer 21 out of 25 questions. Do you think graphing calculators are overused? ”

Potential problems include:  Wording of Questions  Example 2: “ By using a graphing calculator, students in an Algebra I course are able to make visual connection between equations and their graphs, reinforcing difficult concepts. Do you think graphing calculators are overused? ”  Example 3: “ Do you like English or Math? ”  Example 4: “ Do you like school? ”  Wording of Questions  Example 2: “ By using a graphing calculator, students in an Algebra I course are able to make visual connection between equations and their graphs, reinforcing difficult concepts. Do you think graphing calculators are overused? ”  Example 3: “ Do you like English or Math? ”  Example 4: “ Do you like school? ”

Designing Experiments:  If we want to observe individuals and record data without intervention, we conduct an observational study.  If we want to examine a cause and effect relationship, we impose a treatment and conduct an experiment.  If we want to compare the effects of different treatments, one of which is no treatment (control group), we conduct a comparative experiment  If we want to observe individuals and record data without intervention, we conduct an observational study.  If we want to examine a cause and effect relationship, we impose a treatment and conduct an experiment.  If we want to compare the effects of different treatments, one of which is no treatment (control group), we conduct a comparative experiment

Designing Experiments:  The individuals on which the experiment is done are called experimental units.  If the units are people, they are called subjects.  The experimental condition we apply to the units is called the treatment.  The explanatory variables (causing a change in the other variables) are called factors.  The individuals on which the experiment is done are called experimental units.  If the units are people, they are called subjects.  The experimental condition we apply to the units is called the treatment.  The explanatory variables (causing a change in the other variables) are called factors.

Designing Experiments:  The factors may be applied in different levels.  When designing an experiment we want to minimize the effect of lurking variables so that our results are not biased.  Because we may not be able to identify and eliminate all lurking variables, it is essential that we use a control group.  The factors may be applied in different levels.  When designing an experiment we want to minimize the effect of lurking variables so that our results are not biased.  Because we may not be able to identify and eliminate all lurking variables, it is essential that we use a control group.

Designing Experiments:  The control group gets a fake treatment (placebo) to counter the placebo effect and/or any other lurking variables present.  Having a control group allows us to compare the results of the treatments.  The control group gets a fake treatment (placebo) to counter the placebo effect and/or any other lurking variables present.  Having a control group allows us to compare the results of the treatments.

Experimental Design  Step 1: Choose treatments  Identify factors and levels  Control group  Step 2: Assign the experimental units to the treatments  Matching (place similar units in each treatment group)  Randomization (randomly assign units to each treatment group)  Step 1: Choose treatments  Identify factors and levels  Control group  Step 2: Assign the experimental units to the treatments  Matching (place similar units in each treatment group)  Randomization (randomly assign units to each treatment group)

Principles of Experimental Design: 1.Control the effects of lurking variables by comparing several treatments (include a control group if possible). 2.Use randomization to assign subjects/units to treatments. 3.Replicate the experiment on many subjects/units to reduce chance variation in the results. 1.Control the effects of lurking variables by comparing several treatments (include a control group if possible). 2.Use randomization to assign subjects/units to treatments. 3.Replicate the experiment on many subjects/units to reduce chance variation in the results.

Principles of Experimental Design:  Note: An effect is called statistically significant if it is too great to be caused simply by chance.  The concept of “statistical significance” is covered second semester.  Note: An effect is called statistically significant if it is too great to be caused simply by chance.  The concept of “statistical significance” is covered second semester.

Principles of Experimental Design:  Even a well-designed experiment can contain hidden bias, so it is extremely important to handle the subjects/units in exactly the same way.  One way to avoid hidden bias is to conduct a double-blind experiment.  In a double-blind experiment, neither the subjects nor the people who have contact with them know which treatment a subject has received.  Even a well-designed experiment can contain hidden bias, so it is extremely important to handle the subjects/units in exactly the same way.  One way to avoid hidden bias is to conduct a double-blind experiment.  In a double-blind experiment, neither the subjects nor the people who have contact with them know which treatment a subject has received.

Types of Experimental Design:  In a matched pairs design, there are only two treatments. In each block, there is either:  a single subject receiving both treatments, or  a pair of subjects, each receiving a different treatment  In a matched pairs design, there are only two treatments. In each block, there is either:  a single subject receiving both treatments, or  a pair of subjects, each receiving a different treatment

Types of Experimental Design:  In a completely randomized design, all subjects are randomly assigned to treatment groups.  In a block design, subjects are first split into groups called blocks. Subjects within each block have some common characteristic (for example: gender, age, education, ethnicity, etc.) Then, within each block, subjects are randomly assigned to treatment groups.  In a completely randomized design, all subjects are randomly assigned to treatment groups.  In a block design, subjects are first split into groups called blocks. Subjects within each block have some common characteristic (for example: gender, age, education, ethnicity, etc.) Then, within each block, subjects are randomly assigned to treatment groups.

Simulating Experiments:  Question: In a class of 23 unrelated students (no twins!), what is the chance that at least two students have the same birthday? In a room of 41 people? In a room of 50 people?

Simulating Experiments:  To determine the chance of this event, we can: 1.Conduct an experiment and measure the outcomes (Problem: we have to find many classes of the correct size, time, effort) 2.Calculate the theoretical probability using the mathematical laws of probability (Problem: the formulas can be complicated or involve higher mathematics) 3.Simulate an experiment using a model that is similar to this real life event  To determine the chance of this event, we can: 1.Conduct an experiment and measure the outcomes (Problem: we have to find many classes of the correct size, time, effort) 2.Calculate the theoretical probability using the mathematical laws of probability (Problem: the formulas can be complicated or involve higher mathematics) 3.Simulate an experiment using a model that is similar to this real life event

Simulating Experiments:  Using a model to imitate a real life event is called simulation. A well-designed model will yield accurate results for a large number of trials.

Simulating Experiments: Steps Involved in Simulation 1.State the problem or describe the experiment 2.State the assumptions 3.Assign a digit to each outcome 4.Simulate many repetitions 5.State your conclusions Steps Involved in Simulation 1.State the problem or describe the experiment 2.State the assumptions 3.Assign a digit to each outcome 4.Simulate many repetitions 5.State your conclusions

Example: 1.In a class of 23 unrelated students, what is the chance that at least two students have the same birthday? 2.We assume all possible birthdays are equally likely, and that one student ’ s birthday is independent of the other students ’ birthdays. 3.Assign numbers to each day of the year. 4.Randomly choose 23 numbers and look to see if any are duplicated. Record the results in a table.  randInt (1, 365, 23)  L 1  Repeat until you have completed many trials. Example: 1.In a class of 23 unrelated students, what is the chance that at least two students have the same birthday? 2.We assume all possible birthdays are equally likely, and that one student ’ s birthday is independent of the other students ’ birthdays. 3.Assign numbers to each day of the year. 4.Randomly choose 23 numbers and look to see if any are duplicated. Record the results in a table.  randInt (1, 365, 23)  L 1  Repeat until you have completed many trials.

Example: Number of students with the same birthday TallyRelative Frequency None At least 2 Total # of Trials 5.In a class of 23 unrelated students, the chance that at least two students have the same birthday is approximately _______.