Presentation on theme: "The Nature of Science Design of Controlled Experiments"— Presentation transcript:
1 The Nature of Science Design of Controlled Experiments
2 Lesson RationaleBased on personal observations while grading AP Free-Response questions and the debrief which ended the conference, most students have a poor understanding of the design of controlled experiments.
3 Lecture reviews of the: essential nature of science Lesson MilestonesLecture reviews of the:essential nature of sciencegenerally accepted elements of the scientific methodconstruct of a controlled experimentbasic statistical analysis tools from Appendix B: Statistics Primer
4 Lesson MilestonesTasksAccomplishment and review of the 2013 AP Free Response question, Fruit Fly Choice Chamber
5 Science as a means of understanding the natural word Scientific method Key ConceptsScience as a means of understanding the natural wordScientific methodDesigning a Controlled ExperimentStatistics (Chi Square Test) Appendix B
6 Describe what science can do and what it cannot do. Learning ObjectivesDescribe what science can do and what it cannot do.Define the terms observation and data.Define and develop a hypothesis.Explain how scientists test a hypothesis using generally accepted steps to a scientific method.Distinguish between manipulated and responding variables.
7 Explain the terms falsifiable and reproducibility. Learning ObjectivesDistinguish between the colloquial and scientific meanings of the terms theory and law.Explain the terms falsifiable and reproducibility.Calculate a Chi-square analysis and use the result to justify acceptance or rejection of a null hypothesis.
8 Matrix for Planning and Assessing Scientific Inquiry MaterialsMatrix for Planning and Assessing Scientific InquiryAP Free Response, Fruit Fly BehaviorAP Handout of Equations
9 AP Investigations – None Chi-Square Lab AP Investigation/LabAP Investigations – NoneChi-Square Lab
11 How do these pictures relate to each other? DiscussionStudy tying vaccine to autism was fraud, report saysAndrew Wakefield, colleagues altered facts about patients in their research, analysis showsupdated 4 minutes agoLONDON — The first study to link a childhood vaccine to autism was based on doctored information about the children involved, according to a new report on the widely discredited research.The conclusions of the 1998 paper by Andrew Wakefield and colleagues was renounced by 10 of its 13 authors and later retracted by the medical journal Lancet, where it was published. Still, the suggestion the MMR shot was connected to autism spooked parents worldwide and immunization rates for measles, mumps and rubella have never fully recovered.A new examination found, by comparing the reported diagnoses in the paper to hospital records, that Wakefield and colleagues altered facts about patients in their study.The analysis, by British journalist Brian Deer, found that despite the claim in Wakefield's paper that the 12 children studied were normal until they had the MMR shot, five had previously documented developmental problems. Deer also found that all the cases were somehow misrepresented when he compared data from medical records and the children's parents.Called 'an elaborate fraud' Wakefield could not be reached for comment despite repeated calls and requests to the publisher of his recent book, which claims there is a connection between vaccines and autism that has been ignored by the medical establishment. Wakefield now lives in the U.S. where he enjoys a vocal following including celebrity supporters like Jenny McCarthy.Deer's article was paid for by the Sunday Times of London and Britain's Channel 4 television network. It was published online Thursday in the medical journal, BMJ.In an accompanying editorial, BMJ editor Fiona Godlee and colleagues called Wakefield's study "an elaborate fraud." They said Wakefield's work in other journals should be examined to see if it should be retracted.Last May, Wakefield was stripped of his right to practice medicine in Britain. Many other published studies have shown no connection between the MMR vaccination and autism.But measles has surged since Wakefield's paper was published and there are sporadic outbreaks in Europe and the U.S. In 2008, measles was deemed endemic in England and Wales.Copyright 2011 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.How do these pictures relate to each other?
13 Discussion2011 – 118 cases of measles reported in the United States 89% of those (105) patients had not been vaccinated
14 What Science Is and Is Not Science – from the Latin word scientia, meaning “knowledge” Until the Enlightenment (18th century) science was defined as any recorded knowledge
15 What Science Is and Is Not “We may well discover from science many interesting answers to the question ‘How does life work?’What we cannot discover, through science alone are the answers to the questions ‘Why is there life anyway?’ and ‘Why am I here?’ “- Dr. Francis S. Collins, The Language of God
16 What Science Is and Is Not Science – Investigate and understand the natural world, explain events, and use explanations to make useful predictions.Process of reaching an understanding of the natural and physical world.Requires a combination of open- mindedness to new evidence and skepticism.Scientific understanding changes with new discoveries – new evidence.
17 What Science Is and Is Not Three things that set science apart:Deals only with the natural world.Collects and organizes information in an orderly way, looking for patterns and connections between events.Proposes explanations for events that can be tested by experiments.
18 What Science Is and Is Not Faced with a problem or question, scientists seek a solution through testing.What is the process of finding the solution?There is no one way; no one methodBut there are some generally accepted guidelines that constitute what is called the scientific method.
19 A means of gaining knowledge about the natural world by Scientific MethodScientific methodA means of gaining knowledge about the natural world bymaking observations,posing hypotheses, andconducting experiments to test those hypotheses.Glossary, Hillis
20 Scientific observation should be done in an orderly way. Scientific MethodObservationInformation, gathered by the senses, about structures, processes and events.Scientific observation should be done in an orderly way.What can I see, hear, sense in any way?What can I measure?
21 Two categories of data: Qualitative data Quantitative data Scientific MethodObservationData – information; evidence; recorded observations and measurements, either in words or numbers.Accurate data is the evidence that leads to conclusions and useful predictions.Two categories of data:Qualitative dataQuantitative data
22 Descriptions of appearance, impressions, etc., no numbers Scientific MethodQualitative dataDescriptions of appearance, impressions, etc., no numbersInvolves a judgment made by the observer.Quantitative dataExpressed as numbers, obtained by measuring or counting.Mass, weight, number of individuals, size in centimeters, etc.
23 Known as anthropomorphism Scientific MethodQualitative data frequently taken in studies of animal behavior in the wild.The challenge is to describe the animal behavior in an objective manner.Unscientific observers frequently attribute human qualities and motivations to animals.Known as anthropomorphism
24 Let’s say you’ve collected some data. How do you… Scientific MethodLet’s say you’ve collected some data. How do you…Interpret the data and develop an explanation of the event?Form an idea of how and why something works?
25 Develop the tentative idea into a hypothesis In science, a hypothesis Scientific MethodDevelop the tentative idea into a hypothesisIn science, a hypothesisIs a tentative answer to a well-framed question; an explanation on trialA predication that can be testedWe all use hypotheses in solving everyday problems
26 Observations Question Hypothesis #1: Dead batteries Hypothesis #2: LE 1-25aObservationsQuestionHypothesis #1:Dead batteriesHypothesis #2:Burnt-out bulb
27 Test falsifies hypothesis Test does not falsify hypothesis LE 1-25bHypothesis #1:Dead batteriesHypothesis #2:Burnt-out bulbPrediction:Replacing batterieswill fix problemPrediction:Replacing bulbwill fix problemTest predictionTest predictionTest falsifies hypothesisTest does not falsify hypothesis
28 About writing a hypothesis… A valid hypothesis Contains a prediction Scientific MethodAbout writing a hypothesis…A valid hypothesisContains a predictionAnd the predication can be tested in some wayA valid and correct hypothesisIs supported by the evidence gathered by the testThe predication came true!
29 The hypothesis was not supported by the evidence gathered by the test. Scientific MethodThis means that you can have a valid, but ultimately incorrect hypothesis.The hypothesis was not supported by the evidence gathered by the test.
30 Hypotheses are often written in an “If…then” format Scientific MethodHypotheses are often written in an “If…then” formatIf this is done, then the result will be…“If…then” helps the student construct a valid hypothesisOne that contains a prediction
31 Observe, measure, or experiment to test the hypothesis Scientific MethodObserve, measure, or experiment to test the hypothesisDid the prediction come true?Test will help scientists conclude if the evidence will either support or not support the hypothesis.
32 The test of a hypothesis is an experiment. Scientific MethodThe test of a hypothesis is an experiment.Actions or observations taken to test a hypothesisA more defined and structured experiment is known as a controlled experimentAll variables except the variable being tested are kept unchanging, or controlled.
33 Design of a Controlled Experiment An experiment in which a sample is divided into two groups whereby the experimental groups are exposed to manipulations of an independent variable while one group serves are as untreated control.The data from the various groups are then compared to see if there are changes in a dependent variable as a result of the experimental manipulation.Glossary, Hillis
34 Design of a Controlled Experiment An experiment in which a sample is divided into two groups whereby the experimental groups are exposed to manipulations of an independent variable while one group serves are as untreated control.The data from the various groups are then compared to see if there are changes in a dependent variable as a result of the experimental manipulation.Glossary, Hillis
35 Design of a Controlled Experiment All variables, except the variable being tested for the hypothesis,are kept unchanging, controlled, or constantGoal is to make the only difference between the two groupsthe variable whose influence is predicted in the hypothesis
36 Design of a Controlled Experiment VariablesFactors in an experiment that can changeTemperature, light, time, materials, etc.Many variables can be acting at the same timeConstantvariable which is deliberately maintained at a predetermined value during the experimentExample: using an incubator to control temperature while growing bacterial cultures
37 Design of a Controlled Experiment Two critical variables that apply to the hypothesis:Manipulated variable, or independent variable – is deliberately changed.Responding variable, or dependent variable – is being observed and changes in response to the manipulated variable.The hypothesis must connect directly to these two variables!
38 Design of a Controlled Experiment There are also two “groups” involved in a controlled experimentExperimental groupControl group
39 Design of a Controlled Experiment Experimental Group: Group or sample that is experimented onsample that is subjected to the manipulated variableControl Group: Another group or sample that is not experimented on, but in every other way exactly alike to the first groupsample not subjected to manipulated variable that is used as comparison with experimental group.
40 Design of a Controlled Experiment RandomizationIf the design requires sampling of a population – which is likely – such samples must be randomly selected
42 Design of a Controlled Experiment About ready to start the controlled experiment…Will the “If this happens, this should be the result” prediction hold true?Must be able to compare the results of the experimental and control groupsMust show that a phenomena occurs after a certain treatment is given, and it does not occur when the treatment is withheld.
43 Design of a Controlled Experiment Did the data support the predication made by the hypothesis?If yes, hypothesis is validBut are any new questions raised?If no, hypothesis is invalidIs it possible to rework the hypothesis or is an entirely new hypothesis necessary?
44 Design of a Controlled Experiment ConclusionThe place where you got tired of thinking.Valid hypothesis may allow scientists to predict outcomes, or make general statements, for much larger groups.
45 Design of a Controlled Experiment Multiple trials build confidence in resultsConduct the experimental several times using the same procedure.If results are consistent, shows the first time was not a fluke.
46 Design of a Controlled Experiment ReproducibilityAll scientists must be able to do the experiment done by one scientist.Predictions are tested by experiments which can be reproduced by others to see if the same results occur.Publishing and Peer ReviewFellow scientists must be able to critique the experiments and results.
48 Design of a Controlled Experiment Science is not fairNot open to all ideas and explanations.Must be driven by the evidence.
49 Design of a Controlled Experiment Once an explanation is out in the scientific community…frequently finds application to related fields and other similar investigations.
50 A common misconception Truth be said, impossible to keep everything constant except one variable in field experiments. Very difficult for highly regulated laboratory experiments. Control groups also used to cancel the effects of environmental variables that cannot be controlled or eliminated.
51 When (Controlled) Experiments are Not Possible At times your situation does not allow you to test a hypothesis:Technology needed for test does not yet existEthical and legal restrictions on human testingLaboratory setting may cause bias in resultsToo many variables exist; can’t control themIf you want to observe animal behavior in the wild, then observe them in the wild, not the laboratory
52 Let’s discuss the word theory for a moment. How a Theory DevelopsEven though a valid hypothesis exits, the next step is not a brand, new scientific theory.Scientific meaning of the word theory is often misunderstood by students and the general public.Let’s discuss the word theory for a moment.
53 How a Theory DevelopsTheory“A well tested explanation that unifies a broad range of observations.”- Dr. Kenneth Miller
54 How a Theory Develops“All scientific theories represent a framework for making sense out of a body of experimental observations.But the primary utility of a theory is not just to look back, but to look forward. A viable scientific theory predicts other findings and suggests approaches for further experimental verification.”- Dr. Francis S. Collins
55 Is there a process to developing a theory? How a Theory DevelopsIs there a process to developing a theory?A hypothesis or set of hypotheses have been tested – repeatedly and by ideally different investigators – and have not been contradicted by evidence.Now an explanation develops which explains of all or much of the data we have and offers valid predictions that can be further tested.
56 How a Theory DevelopsIn science, a theory can never be proven true, because we can never assume we know all there is to know.Instead, theories remain standing until they are disproved by new evidence, at which point they are thrown out altogether or modified to fit the additional data.
57 Newton’s theory of universal gravitation Cell theory How a Theory DevelopsThere are theories which have such a great body of supporting evidence they are essentially “proven” in the scientific sense:Newton’s theory of universal gravitationCell theoryHeliocentric theoryAtomic theoryCan you ever imagine these theories being proved wrong or falsified?
58 Vital part of science philosophy and practice. How a Theory DevelopsFalsifiabilityA theory is scientific if it leaves open the possibility of being proven wrong or false.Vital part of science philosophy and practice.With the discovery of new evidence, even theories long held may be discarded and replaced by better theories.
59 How a Theory DevelopsFalsifiabilityIt must be possible to make an observation that would show the hypothesis to be false.If an observation is impossible, then falsifiability is not possible. Therefore, the theory or hypothesis is not scientific.
60 Theory: Colloquial Meaning Speculation; a guess How a Theory DevelopsTheory: Colloquial MeaningSpeculation; a guess“Oh, it’s just a theory.”
61 Oh, it’s just a theory…"Evolution is a 'theory', just like gravity. If you don't like it, go jump off a bridge."AnonymousDistinguish between the everyday, colloquial meaning of theory and the scientific meaning. In this case, for over 150 years the theory of evolution has been well supported by the evidence and no other scientifically derived theory has taken its place.
62 How a Theory Develops"It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong" - Richard Feynman
63 How a Theory Develops"The story of a theory's failure often strikes readers as sad and unsatisfying. Since science thrives on self-correction, we who practice this most challenging of human arts do not share such a feeling. We may be unhappy if a favored hypothesis loses or chagrined if theories that we proposed prove inadequate. But refutation almost always contains positive lessons that overwhelm disappointment, even when [...] no new and comprehensive theory has yet filled the void."Stephen Jay Gould ( ), "Bully for Brontosaurus", The Face of Miranda (1991)
64 Sometimes you hear of scientific laws Laws are not: How a Theory DevelopsSometimes you hear of scientific lawsLaws are not:the same as theoriesthe next rung up on the ladder from theories
65 Observations that continue to fit the evidence How a Theory DevelopsLaws areObservations that continue to fit the evidenceNever contradicted by evidenceAppear everywhere in the universeUnchanged since first discovered, since the beginning of the universe and likely will remain soSimple in that they are typically expressed in a simple mathematical equation
66 How a Theory DevelopsFor example, Newton’s Laws of Motion or Boyle’s Gas Law, or the law of conservation of energy
67 Why discuss obsolete theories? History shows that science will discard or change even strongly held theories if new evidence comes to light.Emphasize
68 Video SummationCreative review of the philosophy of science: Symphony of Science – The Poetry of Reality.
69 A final thought on scientific investigation Occam's Razor (also spelt Ockham's Razor), is a principle attributed to the 14th century English logician and Franciscan friar, William of Ockham. In its simplest form, Occam's Razor states that one should make no more assumptions than needed. Put into everyday language, it says The simplest explanation is the best.
70 A final thought on scientific investigation Better said: When multiple explanations are available for a phenomenon, the simplest version is preferred.
73 The Reader’s Perspective The AP Biology ExamThe Reader’s Perspective
74 AP Biology Exam: Reader Perspective Distribute handout, AP Reader Experience Take a moment to read. Review the memo in class.
75 AP Biology Exam: Reader Perspective Distribute the handout, AP Free Response Questions 2013 Take a moment to read quickly through the questions. Now let’s focus on question #1…
76 Question 1: Fruit Fly behavior Drospholia melangaster
77 Question 1: Fruit Fly behavior You did not need to know anything about the fruit fly itself…
78 Question 1: Fruit Fly behavior ..except their size and affinity for sugary solutions. But even without these two facts, it could be answered!
79 Question 1: Fruit Fly behavior What was essential: Clear understanding of the nature of science, and How to set up a controlled experiment! First a few stories, a lesson on controlled experiments, then you’ll answer this question!
80 Question 1: Fruit Fly behavior Distribute worksheet, AP Biology Free Response Question #1, Fruit Fly Choice Chamber 2013
81 Question 1: Fruit Fly behavior ProcessYou will write your answer to subparts (a) through (e) within 22 minutes in classThen a teacher led review of the answerFollowed with peer scoring using the actual scoring rubric and class discussionFinal scoring by teacher after class
82 Question 1: Fruit Fly behavior But wait…There’s more about the fruit fly!Part (d) requires you to perform a Chi- Square test, but you likely do not know what that is…Lets walk through an introduction to this important statistical tool, then we’ll get back to answering the fruit fly free response question.
85 Every controlled experiment must consider the errors that can be made. Sources of ErrorEvery controlled experiment must consider the errors that can be made.Scientific instruments are not perfect; there will always be that “margin of error.”Random events happenAre the samples chosen large enough and/or actually representative of all the subjects?Statistical tools exist to help take into account the error.
86 Chi-square test is one of those tools Should there be a difference between observed results and predicted or expected results, Chi-square allows investigator to judge the influence of simple, random chance.If Chi-square indicates that the difference is likely not due to chance, then the investigator will know to look for other influencing factors.
87 Which alleles go to which gamete? Chi-Square TestExample:Random chance is a major determinant in the inheritance of alleles (different versions of the same gene)Which alleles go to which gamete?Which gamete combines with which gamete?Could other factors or errors be influencing results?
88 We begin with understanding that… Chi-Square TestBiologists need a statistical tool which helps judge the probability that random chance influenced an outcome, such as a genetic cross.We begin with understanding that…Hypothesis leads to certain prediction of data.Experiment results in observed data.
89 What you expect to see based on your hypothesis (predictions) Chi-Square TestThe following equation calculates the likelihood that the difference is due to chance aloneObserved resultsWhat you can observe during the course of an experiment; Data you collectedExpected resultsWhat you expect to see based on your hypothesis (predictions)
90 Can typically be determined from simple rules of probability. Chi-Square TestExpected resultsCan typically be determined from simple rules of probability.Plug into the formula the expected and observed result, determining the Chi-Square value X2Now let’s check our understanding for a moment…
91 Remember what the Chi-Square test is: A statistical test that compares data collected in an experiment (observed) to data that was predicted by the hypothesis and expected as results.If any difference does exist between the observed and expected results, it could simply be due to random chance.We call outcome the null hypothesis.
92 Chi-Square TestNull hypothesis (H0).Any difference between the observed results and the expected results is not significant and due to random chance alone.If the Chi-Square test leads us to accept the null hypothesis, then the difference is due to random chance.If the Chi-Square test leads us to reject the null hypothesis, then the difference is due to another factor and not chance.
93 Let’s say we have a calculated Chi-Square value X2 What’s next? Chi-Square TestLet’s say we have a calculated Chi-Square value X2What’s next?We need to decide an acceptable probability that the observed results, and thereby the Chi- Square value X2, are caused by chance alone.We need to know two thingsDegrees of FreedomProbability value
94 Chi-Square Distribution Table Chi-Square TestChi-Square Distribution TableThe degrees of freedom (df) and probability (p) values are seen in this table.The Chi-Square values are seen in the boxes representing a cross between df and p.
95 Chi-Square Distribution Table Chi-Square TestChi-Square Distribution TableDegrees of Freedom (left column) is determined by one less than the number of possible outcomes in the experiment (df = n – 1). For example, in a coin toss, there are two possible outcomes – heads or tails. Therefore, the df is one (1).
96 Chi-Square Distribution Table Chi-Square TestChi-Square Distribution TableNotice the grey-shaded column. Biologists usually use a probability (p) value of 0.05., which sets the threshold of significance. In other words, it is the threshold for rejecting or accepting the null hypothesis.
97 Chi-Square Distribution Table Chi-Square TestChi-Square Distribution TableAnother way of saying it is this: p=0.05 is setting a hypothesis that the difference between observed data and expected data is entirely due to chance.
98 Chi-Square Distribution Table Chi-Square TestChi-Square Distribution TableIf the Chi-Square value falls within a column with a higher probability value than the 0.05 value, we can say the hypothesis could be correct, at least from a statistical standpoint.
99 Chi-Square Distribution Table Chi-Square TestChi-Square Distribution TableIf the Chi-Square value falls within a column with an equal or lower probability value than the 0.05 value – the 0.01 probability column, then you cannot accept the null hypothesis – something other than random chance is at work.