Presentation is loading. Please wait.

Presentation is loading. Please wait.

Research Design Choices and Causal Inferences

Similar presentations


Presentation on theme: "Research Design Choices and Causal Inferences"— Presentation transcript:

1 Research Design Choices and Causal Inferences
Professor Alexander Settles

2 Approaches and strategies of research design
Quantitative Strategies Exploratory Studies In-depth interviews Focus Group studies Descriptive Studies Survey research Relationship studies Causal Studies Experimental Comparative

3 Approaches and strategies of research design
Qualitative Strategies Explanatory Studies Case Studies Ethnographic studies Interpretive studies Grounded theory Critical Studies Action research

4 Gathering Qualitative Data
Observation Studies Participation Studies Interviewing Studies Archival studies Media Analysis

5 What Tools Are Used in Designing Research?

6 Clear conceptualization
Language of Research Clear conceptualization of concepts Success of Research Shared understanding of concepts

7 What Is Research Design?
Blueprint Plan Guide Framework

8 What Tools Are Used in Designing Research?
MindWriter Project Plan in Gantt chart format

9 Design in the Research Process

10 Descriptors of Research Design
Question Crystallization Perceptual Awareness Data Collection Method Descriptors Purpose of Study Experimental Effects Research Environment Time Dimension Topical Scope

11 Degree of Question Crystallization
Exploratory Study Loose structure Expand understanding Provide insight Develop hypotheses Formal Study Precise procedures Begins with hypotheses Answers research questions

12 Approaches for Exploratory Investigations
Participant observation Film, photographs Psychological testing Case studies Ethnography Expert interviews Document analysis

13 Desired Outcomes of Exploratory Studies
Established range and scope of possible management decisions Established major dimensions of research task Defined a set of subsidiary questions that can guide research design Developed hypotheses about possible causes of management dilemma Learned which hypotheses can be safely ignored Concluded additional research is not needed or not feasible

14 Commonly Used Exploratory Techniques
Secondary Data Analysis Experience Surveys Focus Groups

15 Experience Surveys What is being done?
What has been tried in the past with or without success? How have things changed? Who is involved in the decisions? What problem areas can be seen? Whom can we count on to assist or participate in the research?

16 Descriptors of Research Design
Question Crystallization Perceptual Awareness Data Collection Method Descriptors Purpose of Study Experimental Effects Research Environment Time Dimension Topical Scope

17 Data Collection Method
Monitoring Communication Method of data collection distinguishes between monitoring and communication processes. Monitoring processes are studies in which the researcher inspects the activities of a participant or the nature of some material without eliciting responses from the participant. Examples of monitoring include traffic counts, library searches, and counting cars in a parking lot. In a communication study, the researcher questions the participants and then collects their responses by personal or impersonal means.The collected data may result from 1) telephone or interview conversations, 2) self-administered or self-reported instruments sent through the mail, dropped-off in convenient locations, or transmitted electronically, 3) instruments presented before and/or after a treatment or stimulus condition in an experiment.

18 Descriptors of Research Design
Question Crystallization Perceptual Awareness Data Collection Method Descriptors Purpose of Study Experimental Effects Research Environment Time Dimension Topical Scope

19 The Time Dimension Cross-sectional Longitudinal
Cross-sectional studies are studies conducted only once. They seek to reveal a snapshot at one point in time. Longitudinal studies include repeated measures over an extended period of time. Therefore, longitudinal studies can track changes over time. Despite this advantage, longitudinal studies are expensive and time-intensive.

20 Descriptors of Research Design
Question Crystallization Perceptual Awareness Data Collection Method Descriptors Purpose of Study Experimental Effects Research Environment Time Dimension Topical Scope

21 The Topical Scope Statistical Study Breadth Population inferences
Quantitative Generalizable findings Case Study Depth Detail Qualitative Multiple sources of information

22 Descriptors of Research Design
Question Crystallization Perceptual Awareness Data Collection Method Descriptors Purpose of Study Experimental Effects Research Environment Time Dimension Topical Scope

23 The Research Environment
Field conditions Lab conditions Simulations

24 Descriptors of Research Design
Question Crystallization Perceptual Awareness Data Collection Method Descriptors Purpose of Study Experimental Effects Research Environment Time Dimension Topical Scope

25 Purpose of the Study Reporting Descriptive Casual -Explanatory
Causal -Predictive

26 Descriptive Studies Who? What? How much? When? Where?

27 Descriptive Studies Description of population characteristics
Estimates of frequency of characteristics Discovery of associations among variables In contrast to exploratory studies, more formalized studies are typically structured with clearly stated hypotheses or investigative questions. Formal studies serve a variety of research objectives such as those listed in the slide. The third objective, discovery of variable associations, is sometimes labeled a correlational study, which is a subset of descriptive studies. Correlation is the relationship by which two or more variables change together, such that systematic changes in one accompany systematic changes in the other.

28 Descriptors of Research Design
Question Crystallization Perceptual Awareness Data Collection Method Descriptors Purpose of Study Experimental Effects Research Environment Time Dimension Topical Scope

29 Experimental Effects Ex Post Facto Study
After-the-fact report on what happened to the measured variable Experiment Study involving the manipulation or control of one or more variables to determine the effect on another variable

30 Causation and Experimental Design
Control/Matching Random Assignment

31 Causal Studies Symmetrical Reciprocal Asymmetrical

32 Understanding Casual Relationships
Property Behavior Response Disposition Stimulus

33 Asymmetrical Casual Relationships
Stimulus-Response Property- Disposition Property- Behavior Disposition-Behavior

34 Types of Asymmetrical Causal Relationships
Stimulus-response - An event or change results in a response from some object. A change in work rules leads to a higher level of worker output. A change in government economic policy restricts corporate financial decisions. A price increase results in fewer unit sales. Property-disposition - An existing property causes a disposition. Age and attitudes about saving. Gender attitudes toward social issues. Social class and opinions about taxation. Disposition-behavior Opinions about a brand and its purchase. Job satisfaction and work output. Moral values and tax cheating. Property-behavior -An existing property causes a specific behavior. Stage of the family life cycle and purchases of furniture. Social class and family savings patterns. Age and sports participation.

35 Evidence of Causality Covariance between A and B Time order of events
No other possible causes of B When testing causal hypotheses, we seek three types of evidence: Covariation between A and B. Do we find that A and B occur together in the way hypothesized? When A does not occur, is there also an absence of B? When there is more or less of A, does one also find more or less of B? 2. Time order of events moving in the hypothesized direction. Does A occur before B? 3. No other possible causes of B. Can one determine that C, D, and E do not covary with B in a way that suggests possible causal connections?

36 Descriptors of Research Design
Question Crystallization Perceptual Awareness Data Collection Method Descriptors Purpose of Study Experimental Effects Research Environment Time Dimension Topical Scope

37 Participants’ Perceptional Awareness
No Deviation perceived Deviations perceived as unrelated Deviations perceived as researcher - induced

38 Descriptors of Research Design
Category Options The degree to which the research question has been crystallized Exploratory study Formal study The method of data collection Monitoring Communication Study The power of the researcher to produce effects in the variables under study Experimental Ex post facto The purpose of the study Reporting Descriptive Causal-Explanatory Causal-Predictive The time dimension Cross-sectional Longitudinal The topical scope—breadth and depth—of the study Case Statistical study The research environment Field setting Laboratory research Simulation The participants’ perceptional awareness of the research activity Actual routine Modified routine

39 Readings Review Buchanan, D. A. & Bryman, A. (2007). Contextualizing methods choice in organizational research. Choice of methods does not depend exclusively on links to research aims; A method is not merely a technique for snapping reality into focus; choices of method frame the data windows through which phenomena are observed, influencing interpretative schemas and theoretical development. Research competence thus involves addressing coherently the organizational, historical, political, ethical, evidential, and personal factors relevant to an investigation

40

41 Organizational Properties
Negotiated objectives. Layered permissions. Partisan conclusions. Politics of publishing.

42 Conclusions attributes of the organizational research setting or context, the research tradition or history relevant to a particular study, the inevitable politicization of the organizational researcher’s role, constraints imposed by a growing concern with research ethics, theoretical and audience-related issues in translating evidence into practice, and personal preferences and biases with regard to choice of method.

43 Bergh, D. D. & Fairbank, J. F. (2002)
Bergh, D. D. & Fairbank, J. F. (2002). Measuring and testing change in strategic management research Reliability assumptions of change variables, correlations between the change variable and its initial measure, and selection of unbiased measurement alternatives. Found that the typical approach used to measure and test change (as a simple difference between two measures of the same variable) is usually inappropriate and could lead to inaccurate findings and flawed conclusions.

44 Recommendations Researchers to screen their data in terms of:
reliability of component measures correlation between the component variables equality of component variable variances; and correlation with the initial component variable.

45 Validity and Reliability
Part 2

46 Validity Determines whether the research truly measures that which it was intended to measure or how truthful the research results are.

47 Reliability The extent to which results are consistent over time and an accurate representation of the total population under study is referred to as reliability and if the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable.

48 Methods to Insure Validity
Face Validity- This criterion is an assessment of whether a measure appears, on the face of it, to measure the concept it is intended to measure. Content validity concerns the extent to which a measure adequately represents all facets of a concept. Criterion-related validity applies to instruments than have been developed for usefulness as indicator of specific trait or behavior, either now or in the future. Construct Validity - which concerns the extent to which a measure is related to other measures as specified by theory or previous research. Does a measure stack up with other variables the way we expect it to?

49 Reliability Test-Retest Reliability
Inter-Item Reliability – use of multiple items to measure a single concept. Inter-observer Reliability - the extent to which different interviewers or observers using the same measure get equivalent results.

50 Graphic comparison

51 How can validity be established?
Quantitative studies: measurements, scores, instruments used, research design Qualitative studies: ways that researchers have devised to establish credibility: member checking, triangulation, thick description, peer reviews, external audits

52 How can reliability be established?
Quantitative studies? Assumption of repeatability Qualitative studies? Reframe as dependability and confirmability

53 Reliability Validity Valid Reliable Not Valid Not Reliable Not Valid

54 Methods to insure V&R Triangulation – multiple methods or sources
Controlling for bias Selection bias Observer bias Questionnaire bias Omitted-variable bias Confirmation bias

55 Internal Validity High internal validity means that changes in the dependent variable were caused by—not merely related to or correlated with—the treatment. High internal validity can be attained only in an ideal true experimental design, or when steps have been taken to eliminate confounding factors or influences.

56 Internal Validity Internal validity can be increased by:
Repeating an experiment many times, and by changing or adding independent variables; Switching the control and experimental groups Controlling confounding influences.

57 Internal Validity Controlled experiments have the virtue of producing results with high internal validity, but they have the liability of low external validity

58 External Validity External validity is the extent to which the cause and effect relationship can be generalized to other groups that were not part of the experiment.

59 External Validity External validity is critically important in policy and program evaluation research because we would like our research findings to be generalizable. If external validity is low, then the results are not generalizable.

60 Threats to Validity History confound — Any independent variable other than the treatment variable, that occurs between the pre-test and the post-test.

61 Threats to Validity Maturation confound — Refers to the fact that as the experiment is conducted, people are growing older, and are getting more experienced. (Problem in longitudinal studies in particular)

62 Threats to Validity Testing confound — Refers to different responses that an experimental subject may have to questions they are asked over and over again.

63 Threats to Validity Instrumentation confound — Results from changing measurement instruments. (Example: use of different people to collect data, if there is no inter-rate reliability, you have changed the measurement instrument.)

64 Threats to Validity Regression to the mean — Groups tend to increasingly resemble the mean over time.

65 Threats to Validity Selection bias confound — When self-selection by individuals into groups is possible, a selection bias can result.

66 Threats to Validity confound —
Diffusion of treatment confound — Occurs when a control group cannot be prevented from receiving the treatment in an experiment.


Download ppt "Research Design Choices and Causal Inferences"

Similar presentations


Ads by Google