Presentation on theme: "Complementary Research Methods"— Presentation transcript:
1 Complementary Research Methods Michele Jacobsen, PhDPostdoctoral Fellow, SERNUniversity of CalgaryJacobsen, D. M.
2 OverviewOutline relative merits of Triangulation - the integration of both qualitative and quantitative research methodologiesReview a variety of research methods:Experimental design, ethnography, case study, survey methods.Discuss Nowaczyk and Underwood’s (1995) paper.Jacobsen, D. M.
3 Fundamental Goals... Of Science: Of Scientists: To Understand, To Predict, To ControlOf Scientists:To communicate discoveries and findings to a community of peersJacobsen, D. M.
4 Designing Research Dimensions of Analysis Research Purposes - theoretical or applied?Research Problems - what questions are asked?Research Settings - simulated or natural?Research Investigators - background and trainingResearch Methods - a continuumExperimental, Ethnography, Case study, SurveyJacobsen, D. M.
5 Evaluating Research Validity Reliability A concern for most social scientists is the complex nature of the phenomena under study: human behavior.Multiple perspectives are required in order to adequately reflect the richness of these complexities.ReliabilityConsistency, ReplicabilityUsefulness or Value of InvestigationContribution to knowledgeAdvance THEORY and PRACTICE in disciplineJacobsen, D. M.
6 Research Methodologies A continuum rather than “either/or” QualitativeGoal: To Understand, PredictDescriptive accountsSimilarities and ContrastsApplied and TheoreticalResearch QuestionsField studyNatural conditionsQuantitativeGoal: To Predict and ControlMeasure and EvaluateGeneralize to population, reproductionBasic and TheoreticalHypothesis testingLab studyControlled, contrivedJacobsen, D. M.
7 Validity and Reliability Both Quantitative and Qualitative research designs seek reliable and valid results. For example:Quantitative Reliability: Data that are consistent or stable as indicated by the researcher's ability to replicate the findings.Qualitative: Validity of findings are paramount so that data are representative of a true and full picture of constructs under investigation.Jacobsen, D. M.
8 Part Versus Whole “Whole” is often greater than “Parts” It is a non-trivial matter to infer the behavior of the whole from the behavior of its partsQuantitative research designs strive to identify and isolate specific variables within the context (seeking correlation, relationships, causality) of the study.Qualitative design focuses on a holistic view of what is being studied (via documents, case histories, observations and interviews).Jacobsen, D. M.
9 Data Collection Quantitative Qualitative Emphasis on numerical data, measurable variablesData is collected under controlled conditions in order to rule out the possibility that variables other than the one under study can account for the relationships identifiedQualitativeEmphasis on observation and interpretation.Data are collected within the context of their natural occurrence.Jacobsen, D. M.
10 Static and Dynamic Quantitative Qualitative The accumulation of facts and causes of behavior through careful isolation, measurement and evaluation of variables.Predictability and Control over time.QualitativeConcerned with the changing and dynamic nature of reality.Understanding a Point in timeJacobsen, D. M.
11 Triangulation Combines independent yet complementary research methods. Simultaneous triangulation:Use of both qualitative and quantitative methods at the same timee.g., Survey methods and Case studySequential triangulation:Results of one method are essential for planning the next methode.g., Exploratory Pilot study precedes Experimental designAlso known as “mixed methods”Jacobsen, D. M.
12 Benefits of Triangulation Advantages of each complement the otherresulting in a stronger research design, andmore valid and reliable findings.Inadequacies of individual methods are minimizedthreats to Internal Validity are realized and addressedExampleQuantitative design strives to control for bias so that facts, instances, phenomena can be understood in an objective way.Qualitative approach strives to understand the perspective of participants or a situation by looking at firsthand experience to provide meaningful data.Jacobsen, D. M.
13 Additional BenefitsTriangulation offers a balance between logic and stories.Qualititative research, which emphasizes exploration, understanding, contextualizing, introspection, and theory construction, provides a strong base for wider quantitative measures, scaling, and generalization.Quantitative research, which emphasizes large samples, can provide an overview of an area that can reveal patterns, inconsistencies, and so forth, that can be further investigated with qualitative methods.Jacobsen, D. M.
14 Effectiveness of Triangulation In order for Triangulation to be used effectively, four principles must be adhered to:1. research question(s) must be clearly focused;2. strengths and weaknesses of each chosen method must complement each other;3. data collection methods should be selected according to their relevance to the nature of the phenomenon being studied;4. a continual evaluation of the approach should be under-taken during the study Corner (1990)Jacobsen, D. M.
15 Range of Research Methods Experimental designEthnographyCase studySurveyJacobsen, D. M.
16 Experimental Design Hypothesis testing Independent and Dependent VariablesFor example - Predictor: method of instruction, Resulting differences: math performanceSampling of PopulationExperimental and Controlled ConditionsRandom assignmentJacobsen, D. M.
17 Experimental Research The researcher does something to the subjects or objects or research, and then attempts to determine the effects of these actionsReportingCareful description of sampling procedureInferential statistics, effect size, and so on.Jacobsen, D. M.
18 EthnographyDefined: a picture of the “way of life” of some identifiable group of peopleAnthropology - “doing fieldwork”, “going native”Preoccupied with culture, and how people interact with each otherQualitative Methodology - Both a research process and a productOutcome: an ethnographic accountJacobsen, D. M.
19 Ethnographic ProcessThe ethnographer is the primary research instrumentOne year or more in the field settinglong enough to see a full cycle of activityFor example, a full school yearTension and balance between involvement and detachmentOutsider’s broad and analytical perspective on group studiedInsider view, familiarity, empathy, identification with groupJacobsen, D. M.
20 Field Research Techniques An Inquiry Process of multiple methods:Participant observationprivileged, active participantpassive observerInterviewingkey informants, structured, unstructuredgroups, surveys and questionnairesMaking and using recordshistorical documents, archives, written recordsJacobsen, D. M.
21 Validity and Reliability of the Ethnographic Account “The satisfactoriness of the explanation is what counts, not the power of the method for deriving it”.Significance is derived socially, not statisticallyJacobsen, D. M.
22 Case StudyUnderstanding the intricate complexity, idiosyncrasy of one particular caseinvestigation of a “bounded system”Some entity deemed worthy of close watcha single child, a single classroom, a single school, a single national program…GoalsUnderstand and report the uniqueness of individual cases (both commonalities and differences)Usually no attempt to represent case by single or multiple “scores”Jacobsen, D. M.
23 Case Study Methods Similar to ethnographic field methods Reporting ASKING - InterviewsGather narrative and testimonyWATCHING - ObservationsSEARCHING - Written records and artifactsReportingDevelop a conceptual structure, look for patterns, consistencies, repetitions, and manifestations pertinent to your research question(s)Jacobsen, D. M.
24 Validity and Reliability There are many different stories to be toldDifferent researchers have different questions to answer, different conceptualizations of the situation, and set different boundaries for the caseGeneralizability: What is true of one case is often true about other casesConsistencies can be found - predictabilityHow many cases are needed before patterns emerge? It depends...Jacobsen, D. M.
25 Survey Research Methods Purpose and GoalDescribe specific characteristics of a large group of persons, objects, or institutionsUnderstand present conditions, rather than the effects of particular intervention (as in experimental research)Sample of PopulationGroups of interest are well defined and chosen using well defined rulesRepresentativenessJacobsen, D. M.
26 Survey Methods Mail Telephone Face-to-Face Web-based postage and printing costs, participation rateTelephonesampling, wage and time costs, participation ratesFace-to-Facewage and time costs, participation rates, like structured interviewWeb-basedanytime, anywhere, cost effectiveJacobsen, D. M.
27 Issues in Survey Construction Item (question) and scale constructionPilot Testing and revisionSampling proceduresAnalysis and reporting of resultsGeneralizabilityDrawing conclusions about the conditions, attitudes, opinions, or status of a population of persons, objects, institutions, or other entities.Jacobsen, D. M.
28 Nowaczyk and Underwood (1995) http://epaa.asu.edu/epaa/v3n20.html "Possible Indicators of Research Quality for Colleges and Universities"Used qualitative method, “focus group”, to investigate research quality in higher educationJacobsen, D. M.
29 ResultsAcademics promoted the use of both quantitative and qualitative measures to report on “quality”QUANTITY OF:Journal publications, conference presentations, books and book chapters, awards, grants, budget, and so on…QUALITY OF:Reputation of publication, reputation of granting agency, quality of conference, peer reviews of research programs,…Quality of institutions that hire graduate studentsSocietal benefit of researchJacobsen, D. M.
30 ResourcesJaeger, R. M. (1997). Complementary Research Methods for Research in Education, (2nd ed). American Educational Research Association: Washington, DC.Edyburn, D. L. (1998). The Electronic Scholar: Enhancing Research Productivity with Technology. Prentice-Hall: Columbus, OH.Nowaczyk, R. H., & Underwood, D. G. (1995). Possible indicators of research quality for colleges and universities. Education Policy Analysis Archives, 3(20). [On-line]. Available:Bowen, K. A. (1996). The Sin of Omission -Punishable by Death to Internal Validity: An Argument for Integration of Qualitative and Quantitative Research Methods to Strengthen Internal Validity. [On-line]. Available:Jacobsen, D. M.