Presentation on theme: "Research Methods and Techniques"— Presentation transcript:
1 Research Methods and Techniques John MorrisFaculty of Engineering, Mahasarakham UniversityComputer Science/ Electrical and Computer Engineering,The University of AucklandIolanthe II leaves the Hauraki Gulf under full sail –Auckland-Tauranga Race, 2007
3 Hypotheses Essential to have at least one!! For a large project Several goals several hypothesesorMajor goal split into hypotheses for each stepSeveral competing hypothesesAim is to choose the correct oneKey pointsWell designed experimentsFormal hypothesis
4 Formal hypothesis Guides experimental design Without a formal hypothesisexperiments are ‘random’ and may not show anything!With a formal hypothesisExperiments have a clear aimSupport or refute the hypothesisExperiments must produce a resultSupportorRefutation
5 Formal hypothesis – supported Experiments must produce a resultSupportSupport can be ‘weak’there are alternative explanations for observationsbutyou still have a clear direction .. Design another experiment to support your hypothesis more stronglyStrong supportMay be sufficiently strong to stop!Others will repeat your experiments before your hypothesis is fully acceptedIf necessary, you can strengthen support with more experimentsorRefutationNext slide
6 Formal hypothesis - refuted Experiments must produce a resultSupportorRefutationAbandon initial hypothesisAbandon idea altogetherModify hypothesis based onObservationsFurther literature searchRepeat cycle untilYou have sufficiently strong supportYou consider initial hypothesis and all variants thoroughly refuted
7 Support or ‘proof’Note that I have avoided using ‘proved’ or ‘proof’ in the previous slides andOnly talked about experiments supporting hypothesesVery important scientific principle here!Experiments only support your hypothesesUntil someone does an experiment that refutes it!No experiment proves any hypothesis!Principle 1
8 Support or ‘proof’ When writing up your experiments You should use words likeSupportDemonstrateShowValidatesBut be careful not to imply proof!But neverProve,Proof, etcOnly use prove for a mathematical or logical proofRemember: sometime in the future, someone may do an experiment that refutes (disproves) your hypothesis
9 Support or ‘proof’ Principle 1’s converse is true, however An experiment can disprove a hypothesis!It only takes one valid experiment to refute any hypothesis!Usually, it takes many experiments before your hypothesis is fully accepted by the scientific communityRepeatability principle (see later)Then your hypothesis may be considered to be a basic physical law or principle
10 Support or ‘proof’ Of course .. If you want to refute a widely accepted principle, then you must provide a logical proof that the experiment does show that the principle is not validThis takes courage!Some principles have withstood testing for centuries now!butSome bad theories survived for a long time before they were refutedphlogiston theory survived for more than a century!flat earth theory - several centuriesor more recentlycold fusion - Half life of several years!
11 Random ExperimentsSome great discoveries came from ‘random’ experimentsArchimedes (287 BC – 212 BC)Famous for running naked through the streets of Syracuse yelling “εὕρηκα!”After discovering the principle now known as “Archimedes’ Principle” while having a bathArchimedes is also famous for several other major discoveries!
12 Random ExperimentsSome great discoveries came from ‘random’ experimentsIsaac Newton (1642 – 1727)Famous for discovering gravity when an apple fell on his head leading him to the laws of universal gravitation which explain, among other things, the motion of the planets around the sunNewton also wrote Philosophiæ Naturalis Principia Mathematica ("Mathematical Principles of Natural Philosophy"), 1687, which contains the foundations of classical mechanics and made important contributions to optics
13 Great discoveries Initial observation may have been random but All great discoveries have then been subjected to serious researchNew physical law or theoryHypothesis stating what will happen in accordance with this law if an experiment is madeorExperiments designed to refute hypotheses based on the lawAnalysisAcceptance or rejection
14 Random Experiments Newton’s Laws became the basis of astronomy They explained the motion of the sun, planets, moons, …Thus they have been thoroughly tested by many experiments over the centuriesSome very preciseplanetary perturbations, or the effect of other planets, moons, asteroids, comets, etc, on a planet’s motion, have been predicted well enough to enable the discovery of new planets!Without these experiments, starting with Newton’s Laws as the basis for their hypotheses, we would still be searching for explanations for the motion of the planets ..
15 Scientific MethodDiscovery of the outer planets is a good example of the application of scientific method or a proper approach to researchPrecise measurements of the motion of Uranus showed deviations from Newton’s LawsReject them orPropose a new hypothesisHypothesis: a new planet would explain the deviationsMethod: Use telescopes to examine the sky in the predicted position of the new planetResult: New planet (Neptune) discovered in hypothesized position in 1846Pluto was similarly discovered in 1930 by its effect on the orbits of Uranus and NeptuneNewton’s Laws are still valid!When corrected for relativistic effects
17 I’d like to improve the incomes of cassava farmers in my area AimsYour plan might start out with some grand, but vague, aim ..I’d like to improve the incomes of cassava farmers in my areaThis is a great goal, especially if your family owns a cassava farm, but not (yet) a good basis for research!But there is one key word that will help you form a good planYou need some initial studyRead the literatureTalk to some farmers about their problemsTalk to other engineers doing similar work (eg other crops)Anything else that may give you an idea for a way to impact incomes significantly
18 Refining your aim Your initial goal included the key word incomes So a useful initial study would identifyFactors that affect crop yield (increase tons/ha)Market prices for the cropAll the costs which must be deducted from the market priceSeedFertilizerSoil preparation and planting costsHarvesting costsPost-harvest processing, eg drying and baggingTransport costs… (others)This should suggest a high-cost activity that you believe you can improve
19 Refining your aimThis should suggest a high-cost activity that you believe you can improveMake sure you check the literature at this pointSome things, like fertilizer use, have been extensively studied and it may be hard to make any useful impactDon’t ignore them though ..If it’s a major cost and has a big effect on yield,there is likely still useful work that you can do!Repeating others’ work is also a legitimate part of scientific endeavour butIt is not novel (unless you get a very different result!)It is difficult to publishIt may not meet the criterion for PhD researchYou are expected to do some new and original research
20 Side note: Repeating existing work Repeating others’ work is an important part of scientific endeavourThe repeatability principleResults are not accepted until repeatedbutIt is not novel (unless you get a very different result!)It is difficult to publishIt may not meet the criterion for PhD researchYou are expected to do some new and original researchSo, although it is important ..This type of research is best left until you have a stable job and the time (and support) to test existing work!Industrial and government labs do a lot of this work
21 Side note: Industrial and government labs In industrial labs, we see problems likeAj X and student Y at MSU showed that we can improve yield in trials on 100m2 plots, we need to verify that we can do that in real farms of 100 rai or more ..ie they want to commercialize research done on a small scale in a university for real industrial situationsSo they must repeat the experiments on a larger scaleNo ‘new’ work, just confirming previous hypothesesEssential, but not good material for a PhDSimilarly government labs surveys whole countries to extend limited work done in small trialsEssentially repeating initial small scale experiments done by, for example, graduate students
22 Refining your research plan Let’s assume you have identified field processing as costly and time-consuming: in particularDrying naturally in the fieldSlowAllows insects to eat your cropRain interferesSo now you need an idea to reduce costsForced hot air drying might be fasterDoesn’t give the insects a chanceSo now you have a concrete hypothesisIf we use forced hot air to dry cassava chips, transport costs will be reduced and farmers’ incomes will increase
23 Refining your research plan So now you have a concrete hypothesisIf we use forced hot air to dry cassava chips,transport costs andlosses to insectswill be reduced and farmers’ incomes will increaseNow we can design experiments to test this hypothesis!We can also test it easilyCompare incomebefore using hot air drying toafterwardsCompare lossesbeforeafter
24 Refining your research plan Note that this hypothesis tells you that, before starting any new experiments, you mustMeasure currentIncomeLosses due to insectsOf course, now you need to work outhow to build something to dry the cassavawhat conditions will give the best resultstemperature,air flow,packing of chips,..So you will need to design separate experiments …
25 Refining your research plan So you will need to design separate experiments …For each experiment, you should have a formal hypothesis that it testsThe experiment to determine the best temperature might seem not to need a hypothesis ..“I’ll just test temperatures of 50C, 60C, 70C, 80C and find the best one”However, there is one ..Hypothesis A: “The optimum temperature is 60C”Experiment: Check temperatures near 60C (50,60,70 and 80) to confirm that 60C is the optimumorHypothesis B: “There is an optimum drying temperature”Experiment: Check a range of temperatures (50-80C) to see whether one is better
26 Refining your research plan These are simple experiments,butit always helps to take a minute to formalize the hypothesis you’re testingClearly, if you’re thinking along the correct lines, you can quickly form a hypothesis (see the previous examples)Benefits areYou have an immediate answer because you were clear about your aim (confirm or refute your hypothesis)When results are not clear, the next step is clearYou failed to confirm your hypothesis, soNew hypothesisorNew experiment to test the old one
27 Refining your research plan You may have seen no effect of air temperature on dryingMaybe it actually depends more on original moisture content, chip size or ……….. (some other factor)If you had started withHypothesis B: “There is an optimum drying temperature”A null result (no effect) means you can immediately move to looking at other factors!No need to ponder alternatives!A good hypothesis will often save you time in analysing experimental results
28 Research plan – Design task Assume your research project is to design somethingExample: an underwater autonomous vehicleMy task is clear .. Design an underwater robotWhere’s the hypothesis?I don’t really need one!But you have many ..
29 Research plan – Design task Read the literature and study previous designsLook at problems and limitationsFind something that you can improve upon!Determine requirements for target task egmust be able to operate in rivers flowing at up to 10m/smust be able to operate for at least 3 hours…Design the robotBuild itTest itWrite thesis andGraduateNo hypotheses needed!
30 Research plan – Design task Read the literature and study previous designsLook at problems and limitationsFind something that you can improve upon!Determine requirements for target taskmust be able to operate in rivers flowing at up to 10m/smust be able to operate for at least 3 hours…Design the robotAs soon as we start this phase, we have many implicit hypothesesThe motor and propeller system will meet requirement AThe battery is large enough to meet requirement B
31 Research plan – Design task Read the literature and study previous designsDetermine requirements for target taskDesign the robotAs soon as we start this phase, we have many implicit hypothesesFormalizing these hypotheses guides design of the testsIt also guides writing your report or thesis!An excellent and logical layoutList the requirementsList the hypotheses about the design derived from themYou can devote a section to each hypothesisDesign of that section of the robotDesign of the tests to confirm the hypothesesSimilarly, conclusion summarizes how well each hypothesis was supportedie how well the design aims were met
32 Research proposals Describe work that you’re going to do! Generally have 3 partsIntroduction and backgroundWhat you’re going to doBudgetYou’re preparing this proposal to get some money !!In the ‘what you’re going to do’ sectionList all the hypotheses that you will test in your researchThis is a very clear way of statingYour aims andWhat results you expectClarity is key!If the people giving out money don’t understand what you want to doThey are not likely to support you with their money!
33 Exercise Write down If you think you don’t have a clear topic yet Your research topic (proposed research if you’re just starting!)Aim of this researchWhat do you want to show?This could be very generalA hypothesis associated with this aimSomething more specific that you can design an experiment to testIf you think you don’t have a clear topic yetPropose something NOWIt is OK if it’s a grand aimMay turn out to be impractical laterForm a testable hypothesisThis may show whether it is feasible or impractical!Discuss it with your supervisor later
34 Example aim If you don’t have a research project yet, try this one: I want to do something about Thailand’s deaths from road accidentsI’d like to use vision technology (cameras)Refine this to a feasible research project with some clear hypotheses
35 Today – 1 or 2 pages Name Aim Steps to determine detailed goal Detailed goal (guess results of step 2)One or more testable hypothesesDerived from detailed goalIn English, but no grammar neededWrite lists, use bullet points, ….If you’re not confident in EnglishYou can write in Thai and use Google translator or …Not assessed today .. But your final assessment for this course will require something very similar!!
36 More refinementsReading the literature shows that other researchers have reported benefits from blanching similar starch crops before dryingSo you can form some more hypotheses like:Hypothesis: Blanching chips before drying will reduce drying timeHypothesis: Blanching large chips only is cost effectiveand design experiments to test them
37 Formalizing hypotheses Good hypotheses are based on precise models of the system to be evaluatedPreferably mathematical models: they make it easy topredict resultsmeasure deviationsassess experimental errordetermine the range of values that should be testeddetermine how well the results support the hypothesisExperiments take time and effortTime to design them and make sure they will answer your research question is well spent ..In particular, models should be used to determine the testing domain
38 Example – domain limits Scenario:Modern computers have several caches attached to the CPUProblem: Confirm manufacturer’s claim for cache sizesTypical CPU: L1 cache 16kbytes, L2 cache 2Mbytes, …Access times: L1 cache ~0.6ns, L2 cache ~2ns, ..Hypothesis (basic):Manufacturer’s claims are correctHypothesis (working):If program accesses data set larger than 16kbytes,it will not fit in L1 cache andrunning time will increase significantly because of jump in access time from 0.6ns to 3ns
39 Expected behaviourSharp jump in run timeas L1 cache fillsWrite a small program with some intensive computation step, eg multiply large matrices or FFT on long signal, ..Time the critical loop for several data sizes up to 16kb and overExpect behaviour something like this …Normalized run time4812162024kBData set size
40 Expected behaviourExpected:Sharp jump in run timeas L1 cache fillsTime the critical loop for several data sizes up to 16kb and overExpect behaviour something like this …Actual result ..Barely visible increase in running time!!????Normalized run time*Actual result4812162024kBData set size*Note that the run time was normalized before plotting!Here, this means that if the run time, T(n), for problem size, n, is expected to be T(n) = an2 (for example), then we plot T(n)/n2This point is considered again when discussing presentation of results ……
41 Imprecise model In this case the model is not accurate Sharp jump in run timeas L1 cache fillsIn this case the modelExpect sharp increase in running timeis not accurateCorrect mathematical model shows that the run time increases slowly as data set size goes above 16kbNormalized run timeActual result4812162024kBData set size
42 Correct model In fact, the model is wrong (too simple!) Better model isAccess time, t(ws) = tL1 if ws < 16kb= f tL1 + (1 – f) tL2where fraction of L1 cache accesses (hits), f = 16kb/wsWith this model, t increases gradually (not sharply) as ws increases above 16kbIf we plot t vs ws, it becomes obvious that, to see a clear increase (say a factor of more than 2) in t, we need to run the experiment with much larger wsWith this model, ws = 64kb t(64k) = ~1.6ns compared to t(16k) = 0.6ns soBetter test up to at least ws = 64kb !!Actually this model is also simplistic and ignores somecommon characteristics of data access patterns, so the test program needs more careful design than suggested here ..However, this is not a course on computer architecture, so we’ll skip over this problem!
43 Correct model In fact, the model is wrong (too simple!) This example demonstrates the need forFormal hypothesisExpressed in terms of a mathematical model of the system being testedenablingProper estimates of the domain (or range) of variable parameters to be estimatedBetter estimate of accuracy of resultsIf experimental error is considered, then your results may be consistent with the hypothesis!Always keep in mind the accuracy of any measurement and results computed from it!
44 Summary Good experiments start with a clear aim Express aim as a hypothesis which can be testedStarting with a grand aim is OKBut refine it into testable hypothesesBest hypotheses are based on formal mathematical modelsFirst hypothesis: model is correctDetailed hypothesis: If I measure c with inputs a and b, then I will expect value zUse physical, mathematical, chemical, … theory to predict resultsCalculate expected result(s) before you start!Design experiments to test the model
45 Research Plan Starting with a ‘grand aim’ is fine! Read the literatureMake some preliminary surveys or tests or enquiriesReduce the grand aim to a testable ideaForm a more detailed hypothesisRefine that idea into a number of targetsSeparate hypotheses for which you can design experimentsNote that the hypotheses (and the discussion that generates them) form a natural basis for yourReport, Thesis or PaperSo in forming hypotheses, you have written part of your report already!