RESEARCH DESIGN Vivek Patkar vnpatkar2004@yahoo.co.in

Links between the Basic Elements of Research ONTOLOGY EPISTEMOLOGY METHODOLOGY METHODS SOURCES What’s out there to know? What & how can we know about it? How can we go about acquiring that knowledge? Which precise procedure to use? Which data can we collect?

Research Strategy: A general approach to research determined by the kind of question that the research study hopes to answer Research Design: A general plan for implementing a research strategy. It gives details like whether the study will involve groups or individuals and how many variables Research Procedure: An exact, step-by-step description of a specific research study

Inductive Research Strategy Objective is to establish generalisation (what, why questions) Three principles - Accumulation: knowledge grows by the addition of further well-attested facts. Induction: from valid statements describing observations, valid generalisations can be inferred. Instance confirmation: belief in the validity of generalisation increases with number of instances that have been observed.

Deductive Research Strategy Objective is to test generalisation, hypothesis, theory (how questions, predictions) Put forward a hypothesis that helps building a theory. Deduce conclusions. Examine critically the logic of argument in the theory. Test the conclusions by gathering appropriate data. If data are not consistent with the conclusions, the theory is judged as false, otherwise, temporarily supported (but not proven). [K. Pooper’s Principle of Falsification]

Retroductive Research Strategy Objective is to find mechanism or lay account for explaining (how questions, evaluation) To explain observable phenomena try to discover suitable patterns by constructing a model of them. If model correctly represents the patterns, the phenomena might be causally explained Test further consequences of the model, If tests are successful, existence of the proposed arrangement is acceptable. Repeat the whole model building process to explain known arrangements.

Abductive Research Strategy Objective is to obtain technical account from lay account (exploration, description, understand) Meaning and intentions which direct behaviour of the people is to be captured. Describe this inside view and do not impose outsider views. Turn the lay descriptions of social life to technical descriptions of social life. Develop a theory or theories; it is like grasping the unknown whole from the known parts.

Purposes: What issues to research and why? CONCEPTUAL CONTEXT PURPOSES RESEARCH QUESTION METHODOLOGY VALIDITY Purposes: What issues to research and why? Conceptual Context: What theories will guide the study? Research Question: What is the focus and how various sub-questions are inter-related? Methodology: What techniques to be used? Validity: How one can go wrong? Why should one believe the results obtained?

FOUR PHASES OF RESEARCH Research Interest and Idea: selecting problem & expanding Research Design Phase: 1. specifying concepts and variables in the selected problem 2. operationalising concepts to help measuring variables 3. choice of data collection method - primary data: Survey : questionnaire, interview, observation Experiment, Field study, Case study, Content analysis - secondary data 4. sampling : Probability or Non-probability Empirical Phase: data collection and processing Interpretive Phase: data analysis and report writing FOUR PHASES OF RESEARCH

Design Purpose To provide comprehensive answers to the research questions To control the variables that may influence the research outcome {For example, in a study on voting behaviour and group conformity the variable sex may be controlled by selecting only male subjects for the research} To provide a built-in system to check the factors that may affect validity of the research outcome

Research Design A plan guiding the process of collecting, analysing, validating and interpreting data is called the research design. Helps in determining whether the obtained interpretation can be generalised to a larger population or to a different situation. Aims at eliminating as many alternative explanations for a phenomenon as possible. Depends on the type of research, and time and money available.

Research Design Stages Planning stage: identification, selection and formulation of research problem; formulation of hypothesis and its linkage with theory and literature Selection stage: drawing up the design of the experiment or inquiry, definition and measurement of variables, sampling procedures, tools and techniques of gathering data Operational stage: drawing of the finances and budgeting, recruitment and training of the staff Completion stage: analysis, interpretation and validation of data and report writing

Research Design Elements Types of information and proposed sources that would be tapped. A strategy for gathering and analysing the data. An outline of the procedure that will be employed for validating the results An estimate of the time and cost to complete the study.

Research Design Scope Data gathering strategy Sample size and selection method Instruments that would be used: - questionnaire & interview - passive observations - participative observations - desk study Justification for particular method selection Ethical issues that may be involved and how will they be dealt with, should also be stated.

Constructing a Research Design Influences Research Perspective Biographical research Grounded theory Social representations Theory Epistemology Assumptions Previous Questions Methods Components Audiences and writing Intended generalisation comparisons Sampling Criteria and strategies for quality Triangulation Qual/Qual Qual/Quant Interest in an issue Research Design Doing the research Resources Constructing a Research Design

Features of Good Design Built around a clear research question Well linked to the theoretical background Manageable in resources and time Clear in decision about sampling and selection of the particular methods Sensitive, flexible and adaptive to the field conditions Open to new insights during the research

Research Methodology, Designs & Methods Positivist Methodology (Quantitative Approach) Postpositivist Methodology (Qualitative Approach) Exploratory Designs Descriptive Designs Causal Designs Explanatory Designs Interpretive Designs Critical Designs Herme- neutic Studies Action Res. Studies In-depth Interview Field Surveys Single Factor Exp. Case Studies Ehnog- phic Studies Focus Group Maths. Models Multi- Factor Exp. Semiotic Studies Partici- pative Studies

Exploratory Designs (Quantitative Studies) To get insight about the research problem is the objective (pilot studies) These are small-sample designs They are to help evaluate the importance of variables in the study and drop those marginally related They help in hypothesis formulation and testing Data collection is done from both primary and secondary sources

Descriptive Designs (Quantitative Studies) An extensive picture of the phenomenon is obtained by using a large sample Field studies probe a few issues in-depth Field surveys are broader; more popular Cross-sectional studies are one-shot assessment of a sample of respondents, which could be divided in different classes say by income, education, gender Longitudinal studies use panels of the same participants over a time period to assess the change in response and measure it

Causal Designs (Quantitative Studies) They are relational (how one or more variables are related to each other) or They are experimental to know cause or causes of change in a variable or event (what leads to what) Elaborate tools from statistics labelled as Design of Experiments are employed depending on research problem and environment

Hawthrone Experiment: Randomly chosen few workers of the Western Electric company (Hawthorne, New Jersey, USA) were moved to a special location and subjected to variations in working conditions like lighting, period of rest and speed of machines; surprisingly, performance increased with both positive and negative changes. Reason:- Workers being singled out for attention turned out to be the reason and such change is captured by intervening or confounding variables [Reactivity]

Explanatory Designs (Qualitative Studies) Used primarily for providing description as well as meaning of the phenomenon Prediction is also the goal Use of simple statistical analysis is made for explaining difference in responses from two or more groups

Interpretive Designs (Qualitative Studies) To establish a meaning to an event or social situation is the basic objective Concentrate on norms, standards, rules and values that are held in common and how these all influence human interactions Study of symbols, artifacts, beliefs or meanings attributed by people in the study situation is carried out to describe and explain the phenomenon in greater depth

Critical Designs (Qualitative Studies) To change people’s belief is the overriding objective To bring people to actions that are commensurate with accepted truth and goodness is another aim Being increasingly used in public administration, sociological and education research

Combined Research Designs Archival studies: study of historical records to establish an understanding of the circumstances that characterise an event or period (documents like pamphlets, press releases, annual reports, newspaper articles) Media analysis: use of more current records of different media is done (Content Analysis is employed extensively) Artifact studies: study of objects to gain understanding of culture and values of groups (study of items in the dumping sites, advertisements)

PROPOSAL DESIGN Abstract Introduction Setting Pilot Studies Purposes Conceptual Context Setting Pilot Studies Research Questions Theories, Hypotheses Methodology Methods Logical, Statistical Validity Implications

Steps in Sampling Survey Statement of objective Definition of the population to be studied Determination of sampling frame (listing of all the elements) Determination of sample size Selection of sampling method Sample selection & field work Determination of sampling error S A M P L I N G

Types of Sampling Probability Sampling Non-Probability Sampling Convenience Judgment Quota Snowball Theoretical Simple Random Systematic Stratified Random Cluster Multi-Stage Multi-Phase Successive Sequential Concentration Principle: Focus the sampling on those cases, which are particularly important for the study issue Useful for longitudinal studies

Simple Random Sampling Each element or unit in the entire population has an equal chance of selection. Selection Method: Use the following means to pick up the sample unit till the desired number of units are selected, - Lottery system Random Number Tables It is simple to implement but is time consuming and laborious when the sample size is large. Replacement Method: Selected unit may be put back and could be selected again Non-Replacement Method: Selected unit is not considered again

Systematic Sampling Selection Method: The first unit is selected randomly, but thereafter, the units are selected according to a systematic plan (sampling interval). Systematic sampling will be more precise only if units within the sample are heterogeneous. It is simple in operation but should be used with caution for the population with periodicity. Every 3rd unit If we have 500 people and we want a sample of 50 people then the sampling interval is 500/50 = 10. Thus select a random integer from 1 to 10. Say it is 4, then sampling unit will be 4,14, 24 and so on till 500 to get 50 units.

Stratified Random Sampling Selection Method: The population is divided in non-overlapping homogeneous groups called strata and sample unit is selected randomly from each strata either by equal allocation logic or in proportion to the number of units in a stratum or according to any designated scheme. It provides a better cross-section of the population and brings a more precision in the estimate. Newspaper Readers Urdu Marathi Hindi English Gujarati

Cluster Sampling Selection Method: Divide the population in large number of segments called clusters and select the required number of clusters either by equal or unequal probabilities of selection and all the units in the selected clusters are to be surveyed. Though it is statistically less efficient than the simple random sampling, it is less costly for covering say, a large area.

Non-Probability Sampling Convenience sampling is used in exploratory research for getting a gross estimate of the results, without incurring the cost or time required to select a random sample. Judgment sampling is an extension of convenience sampling. For example, one may decide to draw the entire sample from one "representative" college, even though the population includes all colleges.

Non-Probability Sampling (2) Quota sampling is the non-probability equivalent of stratified sampling. Here the strata and their proportions as they are represent in the population are fixed. Then convenience or judgment sampling is used to select the required number of units from each stratum. Snowball sampling is used when the desired sample characteristic is rare or it is extremely difficult to locate respondents, like in the AIDS studies. Here one element leads to another and so on.

Theoretical Sampling If the aim of the research is to develop a theory, this type of sampling strategy is to be employed Here extent of the basic population and its features are not known in advance Repeated drawing of sampling elements with criteria to be defined again in each step Sample size is not determined in advance Sampling is finished when theoretical saturation has been reached

Categories of Research Design Experimental:- Is there a cause-and-effect relationship between two variables? Quasi-Experimental:- Is there evidence of a cause-and-effect relationship between two variables? (some ambiguity would persist) Non-Experimental:- Is there a relationship between two variables? (no control for confounding) Correlational:- Is there a statistical relation between two variables? (no explanation) Descriptive:- What is the status of individual for a specific group of individuals?

Factorial Experimental Designs Experiment is a study in which one or more independent variables are varied and change in the dependent variable is measured. Thus, It could be of 2X2 or of higher dimension. Validity is checked by employing Analysis of Variance (ANOVA) technique. (Statistical software packages are available to assist this part) Complete Randomisation, Randomised Blocks, Latin Squares & Graeco-Latin Squares are the common frameworks for designing experiments to eliminate bias.

Informal Experimental Designs a) Before & after without control design: Exp. Performance Treatment Performance group before treatment given after treatment (X) (Y) Treatment effect = (Y) – (X)

Informal Experimental Designs b) After-only with control design: Exp. Treatment Performance group given after treatment (Y) Control No Performance group treatment without treatment (Z) Treatment effect = (Y) – (Z)

Informal Experimental Designs c) Before & after with control design: Time 1 Time 2 Exp. Perf. before Treat. Performance group treat. (X) given after treatment (Y) Control Perf. before Treat. Performance group treatment (A) given after treatment (Z) Treatment effect = (Y- X) – (Z-A)

2 X 2 Design : To test the effectiveness of token gifts and follow up reminders alone and in combination on the response rate in a mail survey. No Gift Token Gift No Follow Up 30% (30/100) 45% (45/100) 37.5% (75/200) Follow Up 50% (50/100) 60% (60/100) 55% (110/200) 40% (80/200) 52.5% (105/200) Conclusions: 1. Improvement by follow up : 55% – 37.5 % = 17.5% 2. Improvement by token gift : 52.5% – 40 % = 12.5% 3. Maximum response rate by follow up & token gift (60%)

Some Experimental Designs I C B A D II A B D C III B C D A IV A D C B D A C C B D B A D C B D A B C A Randomised Blocks Complete Randomisation B γ A β D δ C α A δ B α C γ D β D α C δ B β A γ C β D γ A α B δ D B C A B D A C C A D B A C B D Latin Square Graeco-Latin Square

Elaborate Checking Internal checking to safeguard against selecting wrong pattern as true Avoiding faulty generalisation from single event or handful observations Logical testing of arguments and examining the basic assumptions Filling up information gaps Checking for coherence (a causal relation among observations must be established) Contd…

Elaborate Checking (2) Carrying out risk assessment Breaking the creation to examine for any flaw in any of the components, side effects, difficulty in use, changing the environment for variable testing Reviewing by experts Testing through surveys, prototypes, and experimentation

Validity Threats Research Bias: Data selection to fit the theory or preconceptions Reactivity: Influence of researcher on setting of experiments Checks: Modus Operandi Approach [generating alternatives to eliminate] Searching Negative Cases [rule out the possibilities] Triangulation [supporting evidence from diverse sources] Feedback [soliciting observations from experts] Quasi-Statistics [some quantitative support] Comparisons [comparing results from other studies]

Effective Research Design Parsimony: causes advanced as explanations for phenomena be as far as simple as possible (Occam’s razor) Unobtrusiveness: avoids obtrusive data collection Sound Methodology: selects appropriate technique for analysis Triangulation: provides for consistency checks

Research Study Design Steps: Formulation of the research problem. Decision about suitable population for the study and the sampling procedure. Devising techniques for gathering data. Determining the mode of administering the study. Deciding the scheme for editing, coding and processing of data. Indicating the procedures for data analysis. Deciding about the mode of presentation of the research report.

Research Design Decisions What is the study about? What is the purpose of the study and its scope? What are the types of data required? Where can the needed data be found and their sources? What will be the place or area of the study? What periods of time will the study include? What time is approximately required for the study? Contd…

Research Design Decisions What amount of material or number of cases will be needed? What bases will be used for the selection of the material ? What techniques of data gathering will be adopted? What type of sampling, if required, will be used? How will the data be analysed? How best can all these questions be decided upon with minimum resources?

Check List Goal clarification Designing overall study & sampling plan Time Cost Goal clarification Designing overall study & sampling plan Selecting the sample Designing the questionnaire & cover letter Conducting pilot test Revising the questionnaire Printing final questionnaire Locating the sample Mailing & response collection Attempts to get non-respondents Editing and coding the data Data entry and verification Analysing the data Interpreting & validating the analysed data Preparing the report

Research design should incorporate sufficient control & validity checks, but also flexibility to achieve the objective precisely Thank You