1. EXPERIMENT DESIGN

2. Experimental Design Keys to designing project with statistics in mind:
Accurately and clearly define VARIABLES
Accurately define LEVELS OF DATA
Identify the TYPE OF EXPERIMENT (i.e. independent 2-sample, paired 2-sample, 1-sample)
Making sure to use appropriate CONTROLS
Making sure to understand the likely DISTRIBUTION OF DATA
REDUCE EXPERIMENT NOISE (reduce random error introduced by experiment design)

3. Reducing Noise in Results
Experimental Observations = combination
Signal – true effects of variable/outcome
Noise – random error introduced by experimental design
Increase signal-to-noise ratio (decrease noise)
REPEATED MEASUREMENTS (repetition / more trials)
INCREASING THE SAMPLE SIZE (testing more items or individuals to better represent the whole population)
RANDOMIZATION OF SAMPLES (use a “lottery” to assign samples to be tested to different experimental and control groups)
RANDOMIZATION OF EXPERIMENTS (use a “lottery” to assign the order of testing different experimental groups)
REPEATING EXPERIMENTS (replication of an experiment)

4. Quantitative Variables
Technique for increasing the signal-to-noise ratio
What is it?
When is it helpful?
Examples of when to use it
Making repeated measurements
Measuring a single item or event more than once to eliminate error in measuring.  More measurements of a single event lead to greater confidence in calculating an accurate average measurement.
Especially helpful if an individual measurement may have a lot of variability; because it has to be made quickly, it is hard to determine the exact endpoint, or is technically difficult and thus prone to errors.  Does not add value if the measurement is clear-cut, like the answer to a survey question about a person's age or measuring the dimensions of a room in meters.
How many drops of acid does it take to change the color of this indicator solution? Run the reaction several times on aliquots of the same solution.
How long does it take for this specific graphics card to heat the air surrounding it to 100°C? Test the same exact graphics card multiple times.
How long does this turtle spend underwater before surfacing for a breath? Observe the same turtle multiple times.

5. Quantitative Variables
Technique for increasing the signal-to-noise ratio
What is it?
When is it helpful?
Examples of when to use it
Increasing the sample size
Increasing the number of items, or people, that you are collecting data from increases the probability that what you are observing is indicative of the whole population.  Calculations can be made to determine how large the sample size needs to be. See the guide on determining the best sample size for a survey for more details.
Especially helpful when you are trying to draw conclusions about an entire population.  Does not apply if your conclusions are intended to be specific to an individual or single item.
Do teenagers eat healthy foods? Survey a large number of teens, not just five people who always hang out together, about their daily diets.
How do the lung capacities of smokers versus non-smokers compare? Take measurements from many smokers and non-smokers.
How long does a 9-volt (V) battery from brand X power a flashlight? Test multiple manufacturing batches of brand X's 9-V battery.

6. Quantitative Variables
Technique for increasing the signal-to-noise ratio
What is it?
When is it helpful?
Examples of when to use it
Randomization of samples
Using a lottery system to assign samples to different experimental and control groups within a given experiment helps make the starting makeup of the groups as equal as possible, even for variables you might have overlooked.  Some experiments can be completely randomized; other involve blocking first. Blocking allows for the creation of homogenous groups, like males versus females, and then randomization within the block.
Especially critical when the population you are drawing your samples from (which is the population you want to make conclusions about) is very heterogeneous.  May not apply if you need to stratify your population because you want to be able to draw different conclusions about each sub-group. For example, men vs. women in a drug study or different types of resistors in a circuit design.
Which fertilization technique increases crop yield the most? Assign fertilizer treatment to each plot of land by lottery, thus evening out effects of other variables, like soil makeup and water content, among the experimental groups.
Does this medication decrease osteoporosis? Randomly assign people to determining whether a medication is effective. Randomly assign patients to placebo or medication group.

7. Quantitative Variables
Technique for increasing the signal-to-noise ratio
What is it?
When is it helpful?
Examples of when to use it
Randomization of experiments
Using a lottery system to determine the order of carrying out related experiments, rather than relying on an apparently logical order that may introduce other overlooked variables.
Especially critical when you have related experiments from which you are going to draw a single meta conclusion.  Applies to both related experiments done serially using the same equipment, and related experiments done in parallel on different equipment.
Does the length of time plastic is pressed in a mold affect the final strength of the plastic? Rather than running experiments testing 10, 20, 30, etc. seconds of pressing back to back, randomize which length of time is tested first, second, etc. The randomization eliminates potential effects from other variables like different amounts of mixing time for the plastic as the experiments progress and changes to the temperature of the mold over the course of all the experiments.

8. Quantitative Variables
Technique for increasing the signal-to-noise ratio
What is it?
When is it helpful?
Examples of when to use it
Repeating experiments
Repeating an experiment more than once helps determine if the data was a fluke, or represents the normal case. It helps guard against jumping to conclusions without enough evidence.  The number of repeats depends on many factors, including the spread of the data and the availability of resources. Three repeats is usually a good starting place for evaluating the spread of the data.
Repeating experiments is standard scientific practice for most fields. The exceptions are usually when the scale and cost of the experiments make it impossible. For example, drug trials on a rare medical condition, large-scale sociology experiments, and astronomy observations of rare phenomena.
Which wavelength of visible light emits the most heat? Make repeated measurements for each wavelength, randomize the order you conduct the wavelength experiments in, and repeat the entire set of experiments at least twice more on a different days using, if possible, different equipment.