Presentation on theme: "This file contains draft slides which will be updated."— Presentation transcript:
1 This file contains draft slides which will be updated. Research MethodsMichael WoodThis file contains draft slides which will be updated.30 November 2009
2 Reading There are many books available – e.g. Saunders et al (2007) Robson (2002)Easterby-Smith et al (2002)And many others … browse in the libraryThese books vary a lot: some are better on the practical aspects, others on the theoretical aspects. Sometimes you will get different advice from different sources, so you need to consider the rationale behind the advice. Robson is good on most aspects, although Saunders et al is probably more student-friendly
3 Contents Overview of academic business research What must be in a project plan and a project?Formulating research aimsThe design of research projectsEvaluating researchStatistical analysis for researchQualitative data analysisAnalysing data and presenting resultsPhilosophy of researchQuestionnaire designInterview design and qualitative researchReminders about the projectInterviews and qualitative research – more detailMore on literature reviews
4 Overview of academic business research Reading: browse through a book on research methods: e.g. Saunders et al (2007), Robson (2002)These slides intended as a brief summary of the important pointsReread them when you are starting your project
5 Advice on research methods Common sense – don’t forget this!Articles and books reporting similar research – should be discussed in the projectBooks on research methods in generalFocus on chapters relevant to your project.
6 Purpose and characteristics of academic research Discover truth about something; and/orFind a good way of doing somethingMust beSystematic and as thorough and trustworthy as possibleClearly written and with sufficient detail for readers to check credibilityEthical
7 Types of research include … Large scale surveys (of people, organisations, events, etc) analysed statisticallySmall scale surveys with emphasis on “qualitative” detailCase studies (to see how something works in detail)Experiments (change something to see what happens)Models can be set up, tested and used for …Participant observation (observe as participant)Action research (combine research and action)Evaluation… and may other possibilities …be imaginative!Many projects combine several of these
8 Sources of data: many possibilities InterviewsIncluding focus groups, Delphi technique (Robson, 2002:57), various approaches to eliciting comments (e.g. “photo elicitation” – Sam Warren)Questionnaires, including via (be careful …)Documents (minutes of meetings, company reports, etc)The webDatabases – within organisation, of share prices, etcObservations of various kindsEtc …. Be imaginative!Sources of literature is a different issue (Judith’s session is very important for this)
9 Experiments (randomised controlled trials) Put people (or whatever you investigating) in randomly assigned groups, give the groups different treatments, and compare groups to see what differences emerge.Used for testing drugs, diets ( educational methods, different designs for websites, social policies, etc. Lots of examples in Ayres (2007)*.Advantages of experiments over non-interventionist researchDisentangle cause and effect. Can control variables you haven’t even thought of. If done well evidence can be very convincing.Can investigate new things* Ayres, Ian. (2007). Super Crunchers: how anything can be predicted. London: John Murray.
10 But … Experiments are often impractical or unethical Difficulties includeHawthorne effectFailure to assign groups at random (this matters a lot because …)So use less rigorous quasi-experiments instead (Grant & Wall, 2008)* – e.g. in action research you may do a before and after comparison. This is a sort of crude experiment but it is not as convincing as a proper RCT.* Grant, A. M. & Wall, T. D. (2008). The Neglected Science and Art of Quasi-Experimentation: Why-to, When-to, and How-to Advice for Organizational Researchers. Organizational Research Methods (published online, July 18, 2008).
11 Finding a suitable topic InterestCareerFeasibilityUsefulness
12 How to do research Read about topic Draft aims of research. Clear, simple, focused.Draft literature review.Draft research plan – check it is really likely to meet your research aims. Check again.Do research/analysisDraft research/analysis and recommendations/conclusionsCheck it fits together and revise all sectionsIf it doesn’t fit together revise aims and …
13 Practical issues Timing Ethics (remember the form!) Plan this remembering that your supervisor may suggest extensive changes.Gantt chart may help.Ethics (remember the form!)Access to information.Take care: this is often difficult!
14 What must be in a project and a project plan? ReadingProject guidelinesProposal guidelinesSaunders et al (2007), or another similar book
15 What must be in a project? Abstract (short summary of project including conclusions)Background and aims (what you’re trying to find out and why it’s important)Literature review (of relevant previous research which you will build on or extend)Research methods – plan and justification (what you did to meet the aims, and why it was a sensible approach)Analysis (in detail, to convince sceptical readers and impress examiners: important tables, diagrams etc must be in the text, only details in appendix)Results, conclusions, recommendations, limitations, further researchReferences (list works cited in text in alphabetical order)Appendices – Ethics form, extra details for the readerFlexible designs can be more flexible – but everything must be there!
16 Features of a good project Obviously important and interestingDifficult to disagree with becauseArguments and analysis detailed, clear and obviously validPossible objections considered and if possible answeredFits togetherAims met by methods (check this in your project plan)Conclusions follow from analysis
17 References and citations You must give references to publications which you draw on or quoteExact (word for word) quotes must be in “…” and the reference must be givenMaximum about one paragraphUse one of the standard referencing systems – preferably the Harvard (see university website)Copying word for word without “…” and reference is treated as cheating and you will fail!
18 Harvard referencing system Very important to use this (or another established system)Seems easy to me, but causes a lot of difficultyCheck library website (search for Harvard) and/or copy an academic article or book.All references in text like Smith (2001)Then alphabetical list of references at the end. Should include everything referred to, and nothing else.
19 What must be in your project plan (proposal)? See assignment descriptionYou may be able to put parts of it in your project!You should describe and justify your research methods in as much detail as possible
20 Writing style (1) Keep it simple. Short sentences Clear, short paragraphsClear subheadingsRead it through to make sure you can follow it. Swap with a friend and check each others’
21 Writing style (2) I think the EMH was true in this situation… In my opinion the EMH was true …In the author’s opinion the EMH was true …4 The evidence suggests that the EMH was true …5 This shows that the EMH was true …Use 4 or 5.Avoid 1, 2 or 3 because it gives the impression that it’s just your opinion and that other, even wiser, people may see it differently.
22 Writing style (3)I work for … and the problems are … / I interviewed three managers.The author works for … and the problems are … / The author interviewed three managers.Then problems of this organization are … / Three managers were interviewed.Opinions vary here. I (MW) prefer (1). Others prefer (2) or (3).Check with your supervisor.
23 Formulating research aims Reading – most research methods books, e.g. Saunders et al, 2007
24 Research aims or questions Usually start from vague ideaThen formulate a clear aim, or list of aims, that your research will achieve. Think of these as hoped-for outcomes.Alternatively…formulate a clear question or list of questions.This process may require some creative thinkingTechniques like brainstorming and mind maps may be useful
25 Aims, objectives, questions You can formulate your research aims as aims (or objectives if you prefer that word) or questions.These are different ways of saying the same thing. Doesn’t matter which you use, but don’t confuse things by having aims and questionsMay be helpful to have a list or hierarchy of aims, but keep it simple
26 HypothesesHypotheses are statements whose truth you want to test, or “predicted answers” to research questions (Robson, 2002)Occasionally appropriate as a top level research aime.g. to test the hypothesis that “Working at home improves quality of life”Usually best to avoid hypotheses when formulating main research aims because questions or aims tend to be more flexiblee.g. “How does working at home affect quality of life?”Null hypotheses have a (controversial) role in some statistical analysis (… as you will see), but they are not relevant to formulating your overall research aims
27 Research aims or questions Research aims or questions should:Be clearly and simply expressedFit together (so that you have a coherent project)Clarify the intended outcome and scope of the researchYour research aims or questions should alsoBe relevant to your degreeBe achievablePresent a reasonable level of challenge
28 Research aims or questions Must be research aims, not business or personal aims.However, business or personal aims may be part of the background motivating your research aims, and research aims would normally include the aim of making recommendations to people or organisations.Should generally have a limited scope or focus.The danger with general aims is that they lead to superficial research.May relate to theoretical issues. You may be aiming to test, modify or create a theory
29 Theory“Theory” includes models, explanatory frameworks, generalisations, recommendations …Examples ….Your research should link with any relevant theory. It mayUse a theoryDemonstrate that a theory is usefulTest a theoryModify a theory or create a new theory
30 Also ask yourself Is the research worth doing? Are there any ethical or political problems?Is it possible? Have you got access to the necessary data?
31 Is it really going to be useful? What use do you want the results to be? This may be a practical use – to find out how to make more money, or to make life easier – or a contribution to theory, but it should be something that is really worth achieving. Must pass the “so what? test.May help to clarify your aims if you imagine you’ve done the research and write down what you think your conclusions and recommendations might be.Then work backwards from what you want to achieve to the best methods to achieve it.
32 Example of research aims The aims of this research are toDescribe the decision making strategies of small investorsDetermine the effectiveness of these strategiesAny comments? Does this seem reasonable for a Masters project?
33 Another example of research aims The aims of this research project are toEvaluate Method X for planning mountaineering expeditionsIf necessary propose and justify Amended Method X for planning mountaineering expeditions
34 Another example of research aims What are the important quality problems in Company X?How serious are these problems?What is the best strategy for reducing these problems?Any comments? Does this seem reasonable for a Masters project? Does it matter that they are expressed as questions?
35 Three more examples of research aims The aim of this research is to investigate the role of the internet in banking.This research project aims to explain activity based costing.The aim of this project is toTest the efficient market hypothesis for the Athens stock exchange, andDetermine how global warming will influence share prices.Any comments? These are not reasonable for an Masters projects! Why not?
36 Possible research topics Research in a specific organisationBest if they are likely to implement any recommendationsTake care you have adequate access to dataEasier if you have a recognised / paid job there and / or know key players well.Research based on publicly available dataEg share prices, the www, published statisticsResearch based on surveys of the “public”These are just some possibilities. There are more …
37 Design of research projects Design means deciding on the methods and approaches which will best achieve your aimsNeeds thinking out carefully starting from your aimsCheck the proposed design will achieve all your aimsThe design may require the use of a theoretical framework – which should be explained and its use justifiedMay incorporate several approaches (e.g. earlier slide)Some advocate “flexible” designs (E.g. Robson, 2002)E.g. Poppy Jaman’s summary. Any comments?E.g. check aims and designs of these projects.
38 Designing research is not easy! Think about how you can best achieve your aimsConsider all possible types of researchBe imaginativeThink about it again… and againCheck you’ve found the best way you can for meeting all your aims
39 Group exerciseDesign a research plan for one of the projects below, and do a pilot study for part of it. (You may find you need to make the aims / questions more precise.)Michael’s project. The provisional aims are:To evaluate the suitability of the PBS website for prospective PhD studentsTo suggest improvements to the website from this perspectiveAlison’s project on the impact of a Blackberry on family/work-life balance. What are the problems and opportunities, and what would you recommend?… or …
40 projectHow much time do “people” spend on s, is it time “well spent”, and if not how can things be improved?Provisional method: Survey to find how much time is spent on s, and respondents’ opinions on whether this is time “well spent”, and on recommendations (is this a satisfactory method?)And / or other possibilities … ?
41 A general design for a typical Masters degree project If the aim is to find a good strategy to "improve" X in org Y, then a possible design may be:Survey/case studies of Org Y to investigate problems and opportunitiesSurvey/case studies to see how other organisations do X and which approaches work wellBased on (1), (2), the literature, and perhaps creative inspiration, consultations within the organisation, simulation or modelling, devise a strategy likely to improve XTry/test/pilot/monitor the proposed strategy, probably in a limited domain
42 Take care with opinion surveys Suppose your research is about risk management and its effectiveness. You decide to investigate by means of a questionnaire and come up with:70% of people in the organisation think our risk management is unsatisfactory60% think Method X is the best way of improving itYou then present this as the rationale behind your recommendations to improve risk management.But … how do they know?Surely the researcher should find out by rigorous and sensible methods, rather than asking people who may neither know nor care?
43 ExerciseThere are many problems with interviews and questionnaires. Your respondents mayNot know the answersNot understand the questionsBe too lazy to think about the issuesWant to deceive youTry to design the methods for a research project without using interviews or questionnaires. (This is not usually a good idea but it should help you to consider alternatives.)
44 Then …Having designed your research get someone to act as a devil’s advocate and tell youWhat’s wrong with it – why it may fail to deliver what you are aiming forWhat may go wrongWould they trust the answer?
45 Evaluating research Relevant to Planning your own research. Use the following slides toCheck your proposalCheck your final projectCritically reviewing published researchThese slides are intended as a checklist for your research and others’
46 Good research should be: As User-friendly as possibleSimple as possible given the message?As Uncritisable (trustworthy) as possibleTrustworthiness or credibility is particularly important. Can you trust the conclusions? Do you believe them? Are there any flaws? Essential to give readers enough detail to check.As Useful or interesting as possibleClear implications for future? New results?
47 In groups … Choose one of the articles you have been given Assess its User-friendlinessTrustworthiness (pay particular attention to this)UsefulnessBrief feedback session, then we will compare your critiques with my slides
48 Trustworthiness of research: main things to check Each letter represents an issue you should consider
49 JargonMost of these checks are covered by technical jargon, concepts and techniques – e.g. lots of types of validity (internal, external, construct, face …), lots of types of reliability, ideas about test and scale construction (see Robson, 2002), etcRead up only those areas which you think are relevant. I have largely avoided jargon here.Always check sampling – always necessary to consider whether your sample is likely to be representative of your area of interest.
50 Deciding what is Cause and what is effect Important to try to work out what causes what, and how strongly and under what circumstances, so that you know what you should change to achieve a particular effect.Take care – may be more complicated than it appears (ISO 9000 and profitability; drinking and thinking, storks and babies).Variable you haven’t thought of may be the important cause!Experiments (randomised controlled trials) for definitive answers, but may be difficult, so …Quasi-experiments (e.g. a before/after comparison of a trial of a new innovation) insead, but …May be lots of causes. Be suspicious of simple explanations (see Taleb, 2008).
51 Deciding what is Cause and what is effect – more examples A survey of organizations showed that those that used the balanced scorecard were more profitable than those that didn’t.Does this show that the balanced scorecard makes firms more profitable?A survey showed that the average job satisfaction score for a department rose substantially and significantly between 2006 and In 2007 everyone was sent on a week’s computer course in the Seychelles.Would you recommend a computer course for other departments?Does high staff turnover cause poor performance or vice versa? (Glebbeek and Bax, 2004). Does extraversion help people get promoted, or vice versa (Moutafi et al, 2007). Does it matter?What caused the fall of the Berlin Wall?
52 To ensure results Representative … check Sampling Decide what you’re interest in – often called the population or target population.Usually this is too big to look at everything so take a sample. Normally we want the sample to be representative of the population or wider context—so you must check if this is likely.Need to consider how the sample is selected and its size. Badly chosen samples can be biased and give very misleading results.E.g. TV audience research, word length, NRE, non-response bias in surveys, survivor bias in stock price samples
53 How to sample Clarify target population (the whole group of interest) May be a population of people, organisations or …Decide sampling approach. There are many methods of taking a sample from your target population, includingRandomStratifiedPurposiveConvenience (or opportunity, haphazard, accidental)Cluster, snowball, quota, etc (see a book)Decide size of sample – need to balance cost with information obtained. If you analysis is statistical, statistical theory can help …
54 Random samplingMake a numbered list of the target population (a sampling frame)Use random numbers to choose sampleEach member of population has the same chance of being selected (and it’s independent of any biases)Each member of sample selected independentlyIn practice, likely that some members of the sample can’t be found or won’t help, so the sample may be biased. Difficult to deal with this … possibilities …The principle is to ignore all variables and choose at random. This allows for all “noise” variables.
55 Which sampling method?Usually random samples are best for large samples, and purposive samples for small samples analysed qualitatively.Done properly, with a large enough sample, random or stratified samples (probability samples) should be reasonably representative of the population. Can’t assume this about purposive or convenience samples (non-probability samples) because these are selected by factors that are likely to bias the result in one direction or another.
56 Sampling in practiceMany samples are biased and so will not give a good idea of the population – regardless of sample size.E.g. NRE, non-response bias in surveys, survivor bias in …Ideal for large samples is random sampling, but this is often difficult to do properly.E.g. Iraq war death rate (see for another approach), TV audience research.Be suspicious of statistical results from purposive or convenience samplesNeed to be especially careful with small, purposive samples for detailed analysis – consider the purpose and choose accordingly
57 Three surveys to check accuracy of NRE phone service – which is right? A Consumer’s Association survey used a sample of 60 calls, mainly about fares. The worst mistake was when one caller asking for the cheapest fare from London to Manchester was told £162 instead of the cheaper £52 fare which was available via Sheffield and Chesterfield. The percentage correct was …32%A reporter rang four times and each time asked for the cheapest route from London to Manchester. The proportion of the four answers which were correct was25%An NRE sponsored survey found that the answers were97% correct(Source: Breakfast programme, BBC1 TV, April )
58 More sampling problems An MBA student sends out 100 questionnaires to 100 organisations asking if they would be interested in a particular service. Twenty are returned, and of these 6 indicated they may be interested in the serviceThere are 650 firms in the relevant industry sector. How big is the market for the service? Are you sure?Suppose you wanted to find out how common it is for women aged to enjoy running.How would you choose a sample to ask?Other examples and exercises attached
59 Measurements (Indicators) If you want to find out whether customer satisfaction, or quality or profits have improved you must have a sensible way of measuring them.Moreno-Luzon (1993) used managers’ “perceived achievement of objectives” as a measure. Can you see any problems with this?How would you measure quality of service in a casino?How would you check if your proposed measure is valid / reliable / right / accurate?
60 Things to remember with measurements (1) Conventional to distinguish between validity (are you measuring the right thing?) and reliability (consistency)If possible use an existing measurement system (with acknowledgement / permission). This has two advantages – there may be evidence validating it, and you can compare your results with previous results.Remember that some informants may be biased, or too lazy to give good answers, or just ignorant.If possible use triangulation (check with information from different sources)Ask yourself whether your proposed method of measurement really measures the right thing
61 Things to remember with measurements (2) Be especially careful with measures of value. This may have several dimensions (Keeney, 1992)*. E.g. the success of a firm might depend on profitability, worker satisfaction, contribution to the community …If you are measuring the success of a change, remember there may be several different criteria. E.g. …May be useful to use the average (mean) response to a series of questions. Use your common sense to see if this is reasonable, or if they should be kept separate. (See literature on Tests and scales – e.g. Robson, 2002: ).* Keeney, R. L. (1992). Value‑focused thinking: a path to creative decisionmaking. Cambridge, Massachusetts: Harvard University Press.
62 Reliability of measurements Same answer at different times?If anything depends on subjective judgments, check agreement between different judgesEg – marking projectsIf you’re asking a number of questions to get at the same information, check the relationship between answers to these questions – with two questions use a correlation coefficient, with more than two use Cronbach’s Alpha (if you are keen on stats!) – see
64 Theoretical assumptions If the research uses a theory, is the theory right for the purpose? And is it a “valid” theory? (Some theories, of course, are stupid or wrong!) You need a critical evaluation in your literature review.A questionnaire or interview plan may be based on assumptions about what is relevant. Are these assumptions OK?
65 Is the research sufficiently Imaginative? Imagination helpful inThinking of hypotheses to explain things …Thinking of new methods for researching …Thinking of new ways of doing things …Many recommendations for boosting imagination and thinking creatively – e.g.BrainstormingDoing something else and coming back to the tasketc
66 Making sure that you are not being misled by Chance Could your results just be due to chance?Have you taken account of sampling error? (If you repeated your research with another sample are you sure the answer would be the same?)Is the sample large enough?Null hypothesis tests or confidence intervals can be used to answer these questions.Are the measurements reliable?
67 The first CRITIC Cause and effect assumptions OK? Representative sample?Indicators (measurements) OK?Theoretical assumptions OK?Imaginative enough?Chance ruled out as explanation?(Checks needed are mostly common sense – except for Chance.)
68 The second CRITIC C Claim? R Role of the claimant? I Information backing the claim?T Test?I Independent testing?C Cause proposed?Teaching skepticism via the CRITIC acronym and the skeptical inquirerSkeptical Inquirer, Sept-Oct, 2002 by Wayne R. Bartz
69 Two extra checksUse of a devil’s advocate or critical friend. Remember the problem of confirmation bias – you are likely to be more enthusiastic about evidence that confirms your pet ideas than about evidence that undermines it! Get someone to try and be critical and find difficulties with your research – then fix or (if unfixable) discuss the problems.Triangulation – compare results from different sources. Applies to data, methods, observers, theories (Robson, 2002: 174).
70 Anything else…? Is this list complete? Does it address all the flaws you noticed in the paper you looked at?What would you add or change?
71 Checklist: the 3 U’s, the CRITIC and Extra checks User-friendly?UnCRITICisable (trustworthy)?CRITICUseful?Extra checksTriangulationDevils advocate (critical friend)
72 Another measurement problem Andy had answers from lots of questions on a SD, D, N, A, SA scaleHe wanted a measure to tell him which questions produced responses which gave a a clear overall view (COV) from his respondentsHis defined his measurement asCOV = abs(SD+D–A–SA) – N(where SD is the number of SD responses, etc, abs = absolute value)He then highlighted questions for which COV > 0Do you think this is a sensible measurement?
73 Critique of an articleDo you accept what the article says, or are there flaws in the research?Think about the article! Use your common sense.Check the CRITIC.Is it worth publishing? Could you do better?Read round the subject – e.g. other research on the same theme.Would the research benefit from some qualitative work, p values or confidence intervals, case studies, different perspectives, experiments…
75 Qualitative data analysis Aim is detail and depth of understandingDemonstrate and understand possibilities, but not how frequently they occurUse direct quotes (“…”) as evidence and to reduce danger of imposing your perspectiveSometimes may be helpful toSummarise in a table or similarUse coding scheme to analyse statistically (but be careful if the sample is very small!)Further possibilities in Saunders et al, Robson, Thorpe and Holt (2008)
76 Analysing data and presenting results Questionnaires and interview plans, and possibly some data, in appendixGraphs and tables and important quotes from interviewees etc in the main textFocus on your research aims, not the questions in your questionnaireReaders want an analysis which shows how your aims are met. They don’t want to know the answers to all the questions in your questionnaire!Use appropriate summaries – e.g. tables of averages, or of main points from interviews
77 Literature review See Saunders et al (2003) Chapter 3 Focus on relevant books, articles and theoriesBrief on general pointsMore detailed on research of particular relevance to your project – you will need to search for articles using the library databasesCriticalShould lead into your method and analysisMust be properly referenced!
78 Philosophy of research In the textbooks you will find discussions of positivism, social constructivism, phenomenology, etc, etc.In my view, Robson (2002) is the best research methods text for philosophical concepts.Almost all concepts and distinctions here open to serious criticism – see Robson (2002). Most management research articles don’t mention philosophy.I wouldn’t suggest focusing on these ideas unless you are interested – in which case be critical of what you read!If you do want to go into philosophy, use a book like the Penguin Dictionary of Philosophy (Mautner, 2005) or Thorpe and Holt (2008) to check what the words mean.Also note that there are arguments against being prescriptive about research methods and philosophy in books with titles likeAfter method: mess in social science research (Law, 2004), andAgainst method: outline of an anarchistic theory of knowledge (Feyerabend, 1993)
80 Some ideas which are worth mulling over Detailed study of a small sample vs less detailed study of a large sampleInduction vs the Hypothetico-deductive method (Popper) vs Following a framework / paradigm / theory vs DeductionSubjective vs Objective; Facts vs Values
81 Some misguided platitudes The following are often assumed (I think wrongly):There are two distinct kinds of research:Quantitative (=positivist=hypothetico-deductive), andQualitative (=phenomenological=inductive).Instead …Positivist research (only) starts from hypotheses.Instead ...Academics tend to disagree about many of these issues. If you do decide to go into them, please think hard, and don’t accept everything you read in the textbooks uncritically!
82 Qualitative vs quantitative Quantitative usually means statistical – often with largish samplesQualitative means focusing on qualities – usually with smallish samples studied in depthDisadvantage with statistical approaches is that the data on each case is often very superficialDisadvantage with qualitative approaches is that case(s) studied may be untypical and can’t be used for statistical generalisationOften best to use both approaches. This is known as “mixed methods” – search for this keyword in library.This distinction often confused with other distinctions …
83 Regrettable tendency to reduce things to a simple dichotomy If you’re a soft and cuddly person:Soft and cuddly (e.g. interpretivist, qualitative, inductivist …) … is goodHard and spiky (e.g. positivist, quantitative, deductivist, …) … is badBut if you are a hard person you will probably reverse good and bad above. There are really many different dichotomies. Reducing them all to one is neither right nor useful.
84 And …To hard and spiky people, soft and cuddly research is lacking in rigourTo soft and cuddly people, hard and spiky research is naïve and lacking in richness
85 Induction vs hypothetico-deductive method Generalise from the data without preconceptions (induction)Grounded theory. Rigour is in process used to generate theory from dataVersusUse data to test hypotheses or theories (hypothetico-deductive method)Karl Popper. Rigor is in the testing.Theory building vs theory testing is much the same distinction (see Saunders et al, 2007, pp )However, I don’t think these are the only choices …
86 Other useful approaches besides induction and hypothetico-deduction Use a framework or theory or “paradigm” (Kuhn, 1970) to define concepts, questions, and measurements, but without trying to test the theoryArguably what most scientists do most of the time (c.f. Kuhn, 1970). Rigour is in ensuring the theory is a good one, and in using it properly.Deduction from data, theories and framework. E.g. the differences between two quality standards can be deduced.Rigour is in checking the deduction and the info you start withDiffers from the hypothetico deductive method in that the result is the deduction itself, not a confirmation, rejection or revision of a hypothesis or theoryNote that this contradicts the assumption in Saunders et al (2007: ) that there are just two approaches – “deductive” and “inductive”. I think they mean “hypothetico-deductive”, and they omit the two very important possibilities above.
87 An example …How would these four approaches work with a project of interest to you …
88 Karl Popper’s ideas (1)Science works by putting forward bold theories (or hypotheses) and then testing them as thoroughly as possibleProvisionally accept theories that have withstood this testing processTheories must be sufficiently precise to be falsifiable – otherwise not proper science (eg Freud’s theories are too vague…)
89 Karl Popper’s ideas (2) - eg Einstein’s theory of general relativity predicts that light from a distant star will be bent by a small amount by passing close to the sun. Newton’s theory predicts the light will not be bent.Only possible to check during a total eclipse of the sun. In an eclipse in 1918 light was bent as Einstein’s theory predictedNewton’s theory is falsified; Einstein’s lives on and seemed much more credible.
90 Karl Popper’s ideas (3)Theories can come from anywhere – guesswork, intuition, other theories, etcThe process of criticising theories and trying to show they are wrong is vital for scienceThe method applies to both natural and social sciencesHow would you apply Popper’s ideas to a management research project? … in practice, has elements in common with a “critical” attitude …
91 Critical attitude Try to anticipate and discuss criticisms Get a friend to act as a devil’s advocateYour work should be so convincing that it can’t be disputed!Think of any criticisms you have of articles you have read. Make sure the same faults don’t apply to your work.Word “critical” sometimes used in a slightly different, more specific, sense.
92 Questionnaire design: summary Read a (chapter of a) book on questionnairesDevelop a pilot. Remember questionnaires are far more difficult to design than they appear! Check with your pilot respondents:Is it clear?Is it interesting / appealing / user-friendly / not too long? Would you answer it?Does it provide (only) the information you want?Are you still sure a questionnaire is a good idea?
93 Questionnaire design (1) Write down what you want to find outClosed questionsTick boxesRating (Likert) scalesOpen questionsPros and cons of each …Check your questions will enable you to find out what you need to for your research
94 Questionnaire design (2) Covering letterPilot it3-4 nice friendly people to tell you what’s wrong with itPilot the analysis tooConsider sample to send it toAnonymity / confidentialityHow to send it / get it back ( ?)What to do about non-response?
95 Questionnaire design (3) Far too many questionnaires about - many of them very silly. What is the response rate likely to be? Would you fill it in?Are you sure a questionnaire is necessary???Many companies have a policy of not responding to questionnairesAre there any alternatives?Check with your supervisor before sending it out
96 Take care with opinion surveys You can ask someoneWhat she did last weekWhat she does in general termsHer opinion of what she doesWhat she thinks other people doHer opinion of what she thinks other people doHow she thinks things can be improvedWhat she thinks about particular suggestions about how things can be improvedWhat she likes / wants / valuesEtcThink about what type of question you are using and whether it is really useful
97 Interview design: in brief (1) Read a (chapter of a) book on interviewsFollows, or precedes questionnaire, or stands aloneBe clear what you want to find outConsider telephone interviewsSmall sample. Don’t do too many interviews.Plan your questions. Should be open-ended and flexible, and aim for a detailed understandingProbes and prompts
98 Interview design (2) Ask for permission to tape record Transcribe interesting bits to get quotes for your projectGet interviewee relaxed. Anonymity / confidentiality (take care here!)Check you’ve covered everythingSend interviewee transcript afterwards?Some transcripts or parts of transcripts in appendix?
99 Reminders about the project Research aims should be simple, explicit, focussed, motivated, usefulLiterature review should focus on relevance to your projectReferences should be complete and in orderMethods should be the best which are feasible.Analysis chapter should show how hard and skilfully you’ve worked, and why readers should believe you. You need to convince a sceptical reader who may want to know details of how your data was obtained – e.g. source of samples, location of interviews (pub or office?), etc, etc – and analyzed.Conclusions and recommendations should summarise what you have found, and clearly meet the research aims. Also discuss limitations.Changing your mind is to be expected – if necessary rewrite aims after doing the research!
100 Reminders (2) Docs/links at http://userweb.port.ac.uk/~woodm/projects Keep to the 15,000 word limit. You can get a good mark with 13,000 words but not with 17,000 words.Remember the ethics form – no form, no pass!Be particularly careful about NHS ethics clearanceMake use of your supervisor (see Project Guidelines)Plan the timescale (Gantt chart) – allow time for delaysAllow time at the end for your supervisor to read it for you to make any necessary amendmentsIf it’s good, consider publishing a summary in a journal. Ask your supervisor.
101 When starting your project you should … Have a clear aim, and a rough idea of your methods and the relevant literature, and a few ideas about problemsMake an appointment with your supervisor and discuss what you will do and the timescale. Take your proposal and commentsRemember your supervisor may have a holiday planned … agree when you will meet / . Usual to send drafts of chapters when completedRemember the deadline and plan back from this. Send your supervisor a draft of the project at least a month before the deadlineProject guidelines atPractical guidelines on statistical analysis atAny questions to
102 Interviews and qualitative research: more detail I am grateful to Alan Rutter for the next few slides, some of which I have edited slightly
103 Primary data collection: interviewing Useful for accessing peoples’ perceptions, meanings, definitions of situations, eliciting their constructions of reality, etc.Alternative typesstructuredsemi-structuredin-depthEthical considerations
104 Forms of qualitative interviews One to oneOne to manyFocus group interviewsFace to face interviewsTelephone interviewsAfter Saunders, et al, 2000
105 Interview respondents Who will be interviewed and why?How many will be interviewed and how many times?When and for how long will each person be interviewed?Where will each person be interviewed?How will access to the interview situation be organised?
106 Sampling for small sample qualitative research Usually best to use theoretical (purposive) sampling - the selection of individuals who you think will best contribute to the development of a theoryResults apply to immediate situationsMay be tentatively generalised, but the small sample means …
107 Difficulties with interviews Mistrust by respondentse.g. researcher is a management spyLoyalty to organisation/colleaguesAdherence to stereotypical views rather than their own inner feelings and knowledgeComplete indifferenceAn opportunity for respondent to ‘sell’ their ideas
108 Managing the interview Preparation for the interviewthe interview scheduleBeginning the interview - establishing rapportCommunication and listening skillsAsking questionssequence and types of questionsClosing the interview
109 Verifying interview data Body languageMaterial evidencee.g. company/factory tourWriting notesas soon as possible after interviewUse informant verification and secondary sources
110 …Is all research is biased? RememberNeed to demonstrate rigourGood research acknowledges bias and the need to expose it.Devil’s advocates are useful for revealing bias and other problems, but are seldom used.…Is all research is biased?
111 More on Literature reviews I am grateful to Andreas Hoecht for the next 16 slidesDon’t forget the literature should be clearly focused on your research aims, and it should be critical in the sense that you should point out strengths and weaknesses where appropriate
112 Research methods: writing a literature review (Andreas Hoecht) 1.Finding material2. Mapping relevant literatures3. Evaluating literature4.Some practical hints
113 Writing a literature review Finding material There is no prescribed number of sources you should use, it depends on the topicBe wary if you feel that you are drowning in material you found for your topic, it probably means you have not narrowed it down enoughBe wary if you find no sources or very little sources. You normally need some academic sources to be able to write a meaningful literature review
114 What secondary sources should you use? Books:Use textbooks only to get an overview over a topicAcademic monographs (edited books with chapters by different authors) can be very useful. They often explore a topic from different angles or cover different aspects of a topicDon’t use “airport bookstall” books as serious sources
115 Secondary sources Journals: Peer-reviewed academic journal articles should normally be the backbone of your literature reviewThey provide up-to date discussions of topics and are usually more narrowly focused than textbooks“Trade journals” (non peer-reviewed) can provide good introductions to topics and overviews of developments but carry considerably less academic “weight” than academic journals.
116 (Secondary) sourcesSometimes you may be able to find article titles like “ …:A review of the literature” in academic journals. They can save you lots of workInternet:Make sure you are able to distinguish between credible sources and Joe Block’s unsubstantiated viewsReputed organisations’ websites can be good sources of information (but may have a bias/self-interest). (gov. Agencies, internat. Organisations)
117 (Secondary) sources Dissertations and PhDs: Checking dissertations stocked in the library may help you to get a feel for what is expected in a dissertation as well as provide information on a topicGovernment reports/EU reports/other organisations’ reports can be very useful (but are sometimes biased).
118 Searching for literature The key is the use of electronic databasesSome databases are full text (you can download articles directly), others are bibliographic databases (you need to check with library or use inter-library loan requests)Business Sources Premier/Emerald Full Text/Econlit/Science Direct are all recommendedBe patient and creative in the use of keywords
119 Searching for literature CD-Rom newspaper databases (FT, Economist) can be useful toolsFinancial Data and Marketing databases mainly provide primary data
120 Mapping out relevant literatures Don’t put everything you find or everything you read in your literature reviewTime spent on familiarising yourself with and assessing literature for relevance is never wastedOnly after you have gained a good overview over the literature will you be able to decide on your particular “angle” and your research questions
121 Mapping out relevant literature Your database search should tell you how much and what type of literature is availableFor some well-researched topic you will be able to concentrate on the literature directly dealing with your specific topicFor other research ideas, you may need to think about “related areas” or similar experiences in other industries or possible insights from other subject disciplines for enlightenment
122 Mapping out relevant literature An simple example: If you are interested in TQM and small firms you may wish toLook at the TQM literature in general for the pros and cons, constraints and motivesIdentify success and failure factors from the TQM literatureCheck the small business literature for general business conditions and constraintsCheck the small business literature to find out if these success factors apply there
123 Mapping out relevant literature You can draw this as a conceptual map of overlapping circles or as a flow diagram if this suits your learning styleBrainstorming and drawing conceptual maps is best done after you have gained a feel for the literature from your literature search
124 Evaluating literature This becomes easier with experienceWhen reading literature:identify the key arguments that are madeThe reason(s) for the conclusions drawnThey should be either derived from logical deduction (a conclusion following necessarily from premises) and /or based on empirical evidence
125 Evaluating literature Check the logic of the arguments madeDoes this necessarily follow?Check the supporting evidenceIs this data relevant? Is it meaningful and accurate? Could it be interpreted in another way? Which data would I need to challenge this?Check for flaws: tautologies, simplistic analogies, redefinition of terms, moral judgements (ought to)
126 Some practical hintsMake sure you refer to key texts that are frequently cited in the literatureFind out whether there are different “schools” or “camps” in the literature and cover their positions.Use your research questions to structure your literature reviewCheck the validity (logic, empirical evidence) of arguments madeMake clear on what basis you decide to side with a “camp” or author or why you remain unconvinced or oppose a judgement
127 Some practical hintsDon’t overstate your case and be realistic about what you can concludeBe particularly fair to views and arguments you don’t agree with (avoid to be seen as biased)Don’t be shy to critique established “trade names”(academic gurus)Write your literature review for non-specialists and avoid jargonWrite it well structured and easy to read