The scientific method F. Guesdon MED610 DDP March 2013

Is square A darker than B? “Checker shadow illusion”, first described by Adelson, 1995

The “scientific method” model o Describes “best practice” method for scientific discovery o Developed from observation of succesful scientists

History 10 th Century o Ibn al-Haytham (Alhacen): Pioneer in experimental optics and psychology, use of scientific method. 13 th and 14 th Centuries o Bacon (Collection of facts, induction) o Occam (Parsimony) 17 th century o Descartes: Deductive method, predictions o Galileo: Experimental approach

History - Modern 20 th century o Statistical criteria o Popper: Falsification o Kuhn: non-rational aspects

Our learning aims o Reflect on what makes research scientifically sound o Understand what the “Scientific Method” is o Ask if “Scientific Method” really accounts for all scientists need to do

Session plan Problem-solving strategies Case study 1: The law of falling bodies Case study 2: The bacterial origin of peptic ulcer How useful is the “scientific method” model?

Thinking about probabilities o 1% of women have breast cancer (p = 0.01) o If a woman has breast cancer, the probability of a mammogram detecting it is p = 0.8 o If a woman has no breast cancer, the probability of the test being positive is p = 0.1 Estimate the probability that a woman whose mammogram came up positive actually has cancer From Gigerenzer, in The evolution of the mind, Dellarosa Cummins and Allen, Eds, 1988, chapter 1

Probabilistically correct answer o For every 1,000 women tested, 10 will have breast cancer and 990 won’t o Of the 10 women with breast cancer, 8 will be diagnosed correctly by the mammogram o Of the 990 other women, 99 will have falsely positive mammograms o For every 1,000 women tested, = 107 mammograms will be positive o The probability that a positive mammogram indicates a true breast cancer is 8 / 107 = 7.47 %

There are two different ways of thinking

One way thoughts come to mind: Is happy This way of thinking uses perception and intution

Is angry This way of thinking uses perception and intution

Another way thoughts come to mind o Probability of having breast cancer is p c = 0.01 so of 1,000 women, we can expect 1,000 x 0.01 = 10 cases o And therefore 1, = 990 healthy women o The test has a rate of detection of p d = 0.8, so it should pick up 0.8 x 10 = 8 cases from the sample of 1,000 o The test has a false positive rate of p + = 0.1, so 990 x 0.1 = 99 healthy women will also have a positive result o So, there will be in total = 107 positive results. o If I am one of those, the probablility I have cancer is p= 8/107 ≈ 0.075

Type 1 thinking = “natural” o Automatic / intutive / effortless o Uses perception, common sense, training (skills) o Jumps to conclusion o “Heuristic”

Limitations of type 1 thinking o Perception (sensory) biases o We tend to misjudge numerical information o We tend to confuse the most typical with the most probable o We seek solutions that conform with how we perceive a problem rather than how it is objectively (framing, economy of thought)

Type 2 thinking: organised o Based on conscious processing o Rational, analytical o “Unatural”, difficult o Technically accurate o Slow or unconclusive when dealing with complex problems (social / economy etc.)

Common sense relies mostly on type 1 thinking o Provides fast, practical answers o Good for practical problems (hunting, farming, buying and selling, stay safe etc.) o Easy to commmunicate or convince – “Feels right” o Influenced by and produces common knowledge

Contemporary common knowledge o In a given situation, people from different cultures are likley to react differently o In a given situation, people will react differently depending on their personality traits o Women have better verbal skills and more empathy than men

Selecting candidates: Common sense approach o Candidates for PhD position selected by interview o Staff believe they select the best candidates o But they can only judge the performance of students they took in o So how do staff know that they select correctly? o We think common sense works because it “seems” to work

Science developped by mistrusting common sense and organising knowledge

Hypothesis o Focused o Generates specific predictions o Designed to be tested rigorously o Will be rejected as soon as it fails a single test Theory Broad scope Accommodates alternative hypotheses Designed to be inclusive: incorporates as many facts and explanations as possible in a unified framework Will be abandonned if cannot generate good hypotheses, or when a better theory is built

Problem-solving strategies: Common sense v. rational thinking Case study 1: The law of falling bodies Case study 2: The bacterial origin of peptic ulcer How useful is the “scientific method” model?

Hypothesis A proposed relationship between two phenomena Generated for the purpose of predicting future observations Often implies a causal relationship linking the phenomena (mechanism)

Aristotle’s description of the motion of falling bodies The speed of falling objects is proportional to their weights. H H L L

What happens if a light object (L) is tied to a heavy object (H)? 1.The falling speed of the tied objects should be intermediate between those that they would have individually. 2.When tied, the two objects (H+L) form a single object heavier than H, so should fall faster than H alone.

Aristotle’s description can lead to contradictory predictions Limited predictive value Can lead to alternative contradictory predictions Does not explain what it tries to describe

Identifying the problem 1 – Galileo noted the logical inconsistency in Aristotle’s description 2 - Observed that falling objects appear to start slowly and then accelerate 3 – Looked for supporting evidence: dents in a cushion 4 – Seeked to measure how speed increased with time and describe the relation in a manner fully consistent with measurements

How Galileo may have generated his hypothesis He uses the most simple mathematical description of accelerated motion: The speed (V) increases in direct proportion to time (T) since the object was dropped: V =  T

How to test this? The equation V =  T leads to a prediction about distance fallen with time: The distance (D) increases in proportion to time squared (T 2 ): D =  T 2 T = 1 D = α T = 2 D = 4 x α T = 3 D = 9 x α

The rolling ball experiments (1603) o Problem: Free fall is too fast o Solution: Study balls rolling down an inclined beam Gallileo assumed that this motion followed the same law as free fall D1 D2 D3

Simulated Galileo data The data does not fit perfectly the prediction Does that means the hypothesis is wrong? D PredictionData TT2T2 TT2T

Replicating the experiment The data is never perfectly reproducible either Does that mean the experiment is not reliable? D Predicted T 2 Experimental T 2 ♯1♯1 ♯2♯2 … ♯ … … … …14.9

o The most important step when interpreting data is ask if the data is good enough to mean anything. o Many experiments do not give “yes” or “no” answers, just “maybe” answers

Simulated Galileo data The differences are not significant, so the data supports the prediction that D =  T 2 D Predicted T 2 Experimental T 2 ♯1♯1 ♯2♯2 … ♯ … … … …14.9

Value judgments: Interpreting data o A researcher must interpret their data - decide what it means. o Interpretation is informed by controls (standardisation), replication and statistical analysis o But not fully objective, depends on assumptions o The interpretation can be contested by other scientists (peers)

o The most important step when interpreting data is ask if the data is good enough to mean anything. o Many experiments do not give “yes” or “no” answers, just “maybe” answers

The Scientific Method 1.Observe phenomena 2.Develop a hypothesis (inductive thinking) 3.Derive predictions from the hypothesis (deductive thinking). 4.Test one prediction (experiment) 5.Interpret the results: are they consistent with the prediction? If yes, the model passes the test; test another prediction If no, the hypothesis is proven wrong (falsified); alter or discard hypothesis

Inductive reasoning o Imagines possible causes or mechanisms to explain the data o Based on recognition of patterns or trends o Can be intuitive, subjective o Error-prone: risks confusing correlation with causality o Essential to make good hypotheses

Standard model of Scientific Method Hypothesis Data Prediction Test Induction Deduction

Hypotheses are at the core of the scientific method o A hypothesis is an attempt at explaining o Testing a hypothesis is testing our understanding o understanding means being able to make predictions! o This distinguishes investigative science from descriptive science (mapping, cataloguing, sequencing)

Testing hypotheses: Falsification o Experiments must be designed so as to reveal if the hypothesis is wrong o Experiments set up to confirm hypothesis are not informative Karl Popper ( )

Testing to faslsify… How would you test the following hypothesis?

“All cards that have a vowel on one side have an even number on the other side” 4U

Testing the hypothesis o You have a sample of 4 cards: Which card(s) do you need to turn over to test the hypothesis? Write your choice(s) on a piece of paper AJ27

“All cards that have a vowel on one side have an even number on the other side” AJ27 Prediction: Available cards:

Would turning card A test the hypothesis? o What might we find if we turn over card A? 1.An even number 2.An odd number o If it is and odd number, we will have learned that the hypothesis is false

Apply this reasoning to all available cards AJ27 “All cards that have a vowel on one side have an even number on the other side”

Correct choices: Cards A and 7 o If you find an odd number on the other side of A, you will know that the hypothesis is wrong o If you find a vowel on the other side of 7, you will know that the hypothesis is wrong

Were our initial choices wrong? Why? o Card 2 is not informative: whether there is a vowel or consonant on the other side will tell you nothing about the hypothesis - but many people choose it o Most people choose card A but very few people choose card 7 - this shows a natural bias towards seeking confirmation, but ignores half the evidence available

Problem-solving strategies: Common sense v. rational thinking Case study 1: The law of falling bodies 2: The bacterial origin of peptic ulcer How useful is the “scientific method” model?

Pre-1984 view of peptic ulcer o Erosions of the lining of the stomach or duodenum o Believed to be caused by overproduction of stomach acids o Thought to result from lifestyle factors (stress or excess absorption of spicy food) o Treatments were: avoiding lifestyle factors, neutralising stomach acidity or preventing acid secretion by severing nerves o Alleviated symptoms, did not cure the disease

Observations of bacteria o In the 1970s, fiber optic endoscopes made possible stomach biopsies from live patients o Until then, most samples of peptic ulcer tissues had been obtained post mortem o In the 1970s, researchers began to report association of gram-bacillus in 80% of patients with gastric ulcers.

Problems with the new observations o Medical texbooksk asserted that bacteria cannot live in the stomach o The bacteria could be grown in vitro after isolation from the biopsies, preventing detailed characterisation o They were assumed to be Pseudomonas, common contaminants of endoscopes o All this suggested the bacteria seen in ulcer samples were not genuine hosts of the stomach

Flaws in accepted knowledge o Warren noticed that the presence of bacteria in his biopsies strongly correlated gastritis – suggesting an immune reaction against the bacteria o Also, the large numbers of bacteria, their homogeneous distribution and their localisation at the top of the cell layer were inconsistent with accidental contamination

How could Warren’s hypothesis be tested? How would you test / prove the role of bacteria in causing peptic ulcers?

Koch’s postulates 1.The microbe must be found in the bodies of the patients or diseased animals 2.The microbe must be isolated from the patients/ animals and grown outside the body 3.The innoculation of the microbe grown in pure culture should produce the disease in an experimental host 4.The same microbe shoud be re-isolated from the experimental subject after the disease develops

Read following sections of hand-out: o The pilot study (p.2) o Isolating the bacteria (p. 3) o The data (pp. 3-4 – Ignore Fig. 3) o Presenting their results (p.5) Think about questions 4, 1 and 5 (pp.6-7)

Pilot study (pp. 3-4): Design Marshall and Warren recruited 100 patients or healthy volunteers undergoing endoscopy. Each participant had to complete a detailed survey on: o Their symptoms o Their lifestyle histories:  Exposure to animals  Travels  Dental hygiene  Diet (Kentucky Fried Chicken?)

Why these questions?

Aim of the pilot study (1982) o Are there bacteria in normal stomachs? o Does the presence of bacteria correlate with type and severity of pathology? o Can the bacteria be cultured?

Koch’s postulates 1.The microbe must be found in the bodies of the patients or diseased animals 2.The microbe must be isolated from the patients/ animals and grown outside the body 3.The innoculation of the microbe grown in pure culture should produce the disease in an experimental host 4.The same microbe shoud be re-isolated from the experimental subject after the disease develops

Testing postulate 2 (Isolation) o Attempts to grow the bacteria in vitro from 30 different biopsies failed repeatedly o Until an accident happened: Due to an emergency, technical staff once left the petri dishes unattended for 5 days, and were then able to see bacteria. The growth in vitro was too slow for normal 2-days cultures.

Testing postulate 3 (Innoculation) o In spite of repeated attempts, the bacteria grown in vitro did not induce the disease in model animals o In desperation, Marshall subjected himself to a self-experimentation and injested 30 ml of aliquid culture of H. pylori o Seven days later, he became ill

Postulate 4 – re-isolation A silver stain of H. pylori on gastric mucus-secreting epithelial cells of Dr Marshall’s stomach biopsy taken 8 days after he drank a culture of H. pylori.

Problem-solving strategies: Common sense v. rational thinking Case study 1: The law of falling bodies Case study 2: The bacterial origin of peptic ulcer How useful is the “scientific method” model?

Comparing the two case studies o IF Galileo’s study is taken as perfect example of the scientific method, does the study of the causes of pptic ulcer devaites from it?

Is the “Scientific Method” a good model? o Describes the rational element of scientific research o Differentiates science from other disciplines o Does not account for subjective or cultural aspects:  How do scientists decide what to study?  Who decides what to study?  Qualitative (exploratory) research  Role of chance discoveries

Pitfalls of the scientific method Complex phenomena cannot always be understood – predicted - through simple hypotheses Examples: o Complex interactions between multiple factors o Phenomena involving non-linear responses to small changes o Common chronic diseases, weather, climate

“Science is what scientists do, and there are as many scientific methods as there are individual scientists.“ P.W. Bridgman (Nobel Prize in Physics 1923)

Alternatives to the scientific method 1 - Exploratory research (mapping, systematic fact finding) coupled with pattern- finding approaches El niño Genome-wide associations with diseases, identification of disease markers) 2 – Modelling

Any questions?