Company LOGO Types of research Jan Štochl

Content 2. Types of research designs 3. Experimental design 4. Quasi-experimental design 5. Nonexperimental design 1. Introduction to research

What is research? Research is an active, diligent and systematic process of inquiry in order to discover, interpret or revise facts, events, behaviours, or theories, or to make practical applications with the help of such facts, laws or theories.

Phases of research Phase 1 Phase 2 Phase 3 1.Problem 2.Studiing recent publications 3. Decision on the problem 4.Determine design and methods 5.Organisation 1.Taking a sample from population 2.Pilot study 3.Correction of errors 4.Data gathering 1.Data analysis 2.Interpretation of results 3.Discussion 4.Publication 5.(Crossvalidation)

Random assignment The assignment of individuals in the pool of all potential participants to either the experimental (treatment) group or the control group in such a manner that their assignment to a group is determined entirely by chance It is an attempt to try to minimize effects of random variables by distributing them randomly across groups Individuals are placed into groups or treatment conditions in such a way that each person has an equal chance of being selected for any group or treatment. In addition, placement of any individual into a group or treatment condition does not influence the placement of any other person

Types of research

What is Research Design? Research design can be thought of as the structure of research -- it is the "glue" that holds all of the elements in a research project together Design is described using a concise notation that enables us to summarize a complex design structure efficiently the "elements" that a design includes: –Observations or Measures (O) –Treatments or Programs (X) –Groups (lines) –Assignment to Group (R = random assignment; N = nonequivalent groups; C = assignment by cutoff) –Time (goes from left to right)

Examples of research design

Examples of experiment, quasi-experiment and nonexperiment Can you „read“ it?

Experimental design Experimental designs are often touted as the most "rigorous" of all research designs or, as the "gold standard" against which all other designs are judged Experiment is probably the strongest design with respect to internal validity internal validity Can „prove“ cause-effect relationship

Establishing cause-effect Three major conditions have to be met: 1.Temporal Precedence 2. Covariation of the Cause and Effect 3. No Plausible Alternative Explanations

Internal validity Internal Validity is the approximate truth about inferences regarding cause-effect or causal relationships relevant in studies that try to establish a causal relationship not relevant in most observational or descriptive studies Zpět

Experimental design simultaneously address the two following propositions: If X, then Y and If not X, then not Y Or, once again more colloquially: If the program is given, then the outcome occurs and If the program is not given, then the outcome does not occur

Mill´s methods 1. The Joint Method of Agreement and Difference 2. The Method of Agreement 3. The Method of Difference 4. The Method of Residues 5. The Method of Concomitant Variations

Classifying experimental design Signal enhancers Noise reducers

Signal-enhancing experimental designs Also called factorial designs

Possible outcomes of factorial designs

Statistical analysis of factorial experimental designs

Features of factorial experimental design Great flexibility for exploring or enhancing the “signal” (treatment) in our studies Efficiency The only effective way to examine interaction effects.

Noise-reducing experimental designs Two major types: 1.covariance designs 2.blocking designs

Covariance designs Also called analysis of covariance design (ANCOVA) Example: simple randomized two group pretest postest design measure is not necessarily the same

Basic idea behind noise reduction

Statistical analysis of covariance designs The so-called ANCOVA

Summary of covariance designs It "adjusts" posttest scores for variability on the covariate (pretest). This is what we mean by "adjusting" for the effects of one variable on another any continuous variable can be used as a covariate, but the pretest is usually best. (Because the pretest is usually the variable that would be most highly correlated with the posttest) Because it's so highly correlated, when you "subtract it out" or "remove' it, you're removing more extraneous variability from the posttest.

Randomized block designs It is a research design's equivalent to stratified random samplingstratified random sampling Useful if some homogeneous groups are recognised in the sample Researcher divide the sample into relatively homogeneous subgroups or blocks The key idea is that the variability within each block is less than the variability of the entire sample. Each estimate of the treatment effect within a block is more efficient than estimates across the entire sample When we pool these more efficient estimates across blocks, we should get an overall more efficient estimate than we would without blocking.

Example of randomized block design

Statistical analysis of randomized block designs Regression analysis

Hybrid experimental designs New strains that are formed by combining features of more established designs The Solomon Four-Group Design Switching Replications Design

The Solomon Four-Group Design Deals with a potential testing threat. A testing threat occurs when the act of taking a test affects how people score on a retest or posttest.

The Solomon Four-Group Design – possible outcomes

Switching replications One of the strongest experimental design

Switching replications – possible outcomes

Two-group experimental design The simplest of all experimental designs is the two-group posttest-only randomized experiment Typically we measure the groups on one or more measures and we compare them by testing for the differences between the means using a t-test or one way Analysis of Variance (ANOVA).

Two-group experimental design The simplest of all experimental designs The posttest-only randomized experiment is strong against the single-group threats to internal validity because it's not a single group design!single-group threats It's strong against the all of the multiple-group threats except for selection-mortalitymultiple-group threats It is susceptible to all of the social interaction threats to internal validitysocial interaction threats

Statistical analysis of two group design The so-called t-test – assessing the mean difference

Single group threats Zpět

Multiple groups threats Selection-History Threat Selection-Maturation Threat Selection-Testing Threat Selection-Instrumentation Threat Selection-Mortality Threat Selection-Regression Threat Zpět

Social threats Diffusion or Imitation of TreatmentCompensatory Rivalry Resentful DemoralizationCompensatory Equalization of Treatment Zpět

Quasi-experimental designs Looks like an experimental design but lacks the key ingredient -- random assignment With respect to internal validity, they often appear to be inferior to randomized experimentsinternal validity Nonequivalent qroups design and regression-discontinuity design

Nonequivalent groups design Most frequent in the social research Identical to the Analysis of Covariance design except that the groups are not created through random assignment This nonrandom assignment complicates the statistical analysis a lot susceptible to the internal validity threat of selectioninternal validity threat of selection

The regression-discontinuity design C indicates that groups are assigned by means of a cutoff score All persons on one side of the cutoff are assigned to one group; all persons on the other side of the cutoff are assigned to the other Need a continuous quantitative pre-program measure

Logic of regression-discontinuity design

Selection of cutoff and interpretation of results

Other quasi-experimental designs The Proxy Pretest Design The Separate Pre-Post Samples Design The Double Pretest Design The Switching Replications Design The Nonequivalent Dependent Variables (NEDV) Design The Regression Point Displacement (RPD) Design

Relationships between pre-post designs

Minimizing threats to validity By Argument By Measurement or Observation By Design By Analysis By Preventive Action

Design Construction - Basic Design Elements 1.Time 2.Program(s) or Treatment(s) 3.Observation(s) or Measure(s) 4. Groups or Individuals

Advances in Quasi-Experimentation The Role of Judgment The Case for Tailored Designs The Crucial Role of Theory Attention to Program Implementation The Importance of Quality Control The Advantages of Multiple Perspectives Evolution of the Concept of Validity Development of Increasingly Complex Realistic Analytic Models

Nonexperimental design Observation Questionnaire Interview Rating scale Correlation study Nonexperimental design Cannot prove causation!!!

Survey research Involves any measurement procedures that involve asking questions of respondents Can be anything from short paper-and- pencil feedback form to an intensive one- on-one in-depth interview Usually surveys are divided into two broad categories: interviews and questionnaires

Types of questionnaires mail survey group administered questionnaire household drop-off survey

Types of interview personal interview telephone interview

Selecting the survey method - population issues What follows is the set of considerations with population and its accesibility: Can the population be enumerated? Is the population literate? Are there language issues? Will the population cooperate? What are the geographic restrictions?

Selecting the survey method - sampling issues What data is available? Can respondents be found? Who is the respondent? Can all members of population be sampled? Are response rates likely to be a problem?

Selecting the survey method - question issues What types of questions can be asked? How complex will the questions be? Will screening questions be needed? Can question sequence be controlled? Will lengthy questions be asked? Will long response scales be used?

Selecting the survey method - content issues Can the respondents be expected to know about the issue? Will respondent need to consult records?

Selecting the survey method - bias issues Can social desirability be avoided? Can interviewer distortion and subversion be controlled? Can false respondents be avoided?

Selecting the survey method - administrative issues Costs Facilities Time Personnel

Constructing the survey Survey questions can be divided into two broad types: structured and unstructured. From an instrument design point of view, the structured questions pose the greater difficulties From a content perspective, it may actually be more difficult to write good unstructured questions

Types of questions Dichotomolus questions

Types of questions Questions based on level of measurement Purely nominal Ordinal like

Types of questions Likert response scale Semantic differential

Types of questions Cumulative or Guttman scale

Types of questions Filter or Contingency Questions

Interviews Another type of survey design From the psychological point of view the most difficult method Usually personal or phone

The role of interviewer Locate and enlist cooperation of respondents Motivate respondents to do good job Observe quality of responses Conduct a good interview

Training the interviewer major topics that should be included in interviewer training: 1)State who is sponsor of research 2)Describe the entire study 3)Teach enough about survey research 4)Explain interviewer bias 5)….

Interviewer kit A "professional-looking" 3-ring notebook (this might even have the logo of the company or organization conducting the interviews) Maps Sufficient copies of the survey instrument Ifficial identification (preferable a picture ID) A cover letter from the Principal Investigator or Sponsor A phone number the respondent can call to verify the interviewer's authenticity

Pluses and minuses of survey research

Qualitative research The purpose of this section is to introduce you to the idea of qualitative research (and how it is related to quantitative research) and give you some orientation to the major types of qualitative research data, approaches and methods.

Qualitative research considerations Do you want to generate new theories or hypotheses? Do you need to achieve a deep understanding of the issues? Are you willing to trade detail for generalizability?

The Qualitative-Quantitative Debate Quantitative research excels at summarizing large amounts of data and reaching generalizations based on statistical projections. Qualitative research excels at "telling the story" from the participant's viewpoint, providing the rich descriptive detail that sets quantitative results into their human context. Necessity of mixed approach

Qualitative and Quantitative Data All qualitative data can be coded quantitatively All quantitative data is based on qualitative judgment

Qualitative and Quantitative Assumptions Many people think that: Quantitative research is confirmatory and deductive in nature. Qualitative research is exploratory and inductive in nature. This can be however misleading

Qualitative and Quantitative Assumptions Differences are based more on epistemological assumptions, i.e. many qualitative researchers believe that the best way to understand any phenomenon is to view it in its context. Many qualitative researchers also operate under different ontological assumptions about the world. They don't assume that there is a single unitary reality apart from our perceptions.

Qualitative data In-Depth Interviews – includes both individual interviews (e.g., one-on-one) as well as "group" interviews (including focus groups). Direct Observation – it is meant in a very broad sense. Differs from interviewing in that the observer does not actively query the respondent. Written Documents - It can include newspapers, magazines, books, websites, memos, transcripts of conversations, annual reports, and so on.

Qualitative approaches Ethnography-studying an entire culture Phenomenology-people's subjective experiences and interpretations of the world Field Research-the researcher goes "into the field" to observe the phenomenon in its natural state Grounded Theory

developed by Glaser and Strauss in the 1960s The purpose of grounded theory is to develop theory about phenomena of interest

Grounded Theory – key strategies Coding is a process for both categorizing qualitative data and for describing the implications and details of these categories. Memoing is a process for recording the thoughts and ideas of the researcher as they evolve throughout the study Integrative diagrams and sessions are used to pull all of the detail together, to help make sense of the data with respect to the emerging theory.

Qualitative Methods Participant Observation Direct Observation Unstructured Interviewing Case Studies

Random selection Probability sampling method is any method of sampling that utilizes some form of random selection It is necessary to set up some process or procedure that assures that the different units in your population have equal probabilities of being chosen. Past - picking a name out of a hat, or choosing the short straw Present – computers and random numbers

Random selection versus random assignment Random selection is how you draw the sample of people for your study from a population. Random assignment is how you assign the sample that you draw to different groups or treatments in your study

Simple random sampling Objective: To select n units out of N such that each N C n has an equal chance of being selected Procedure: Use a table of random numbers, a computer random number generator, or a mechanical device to select the sample.

Stratified Random Sampling Objective: also sometimes called proportional or quota random sampling, involves dividing your population into homogeneous subgroups and then taking a simple random sample in each subgroup Procedure: Divide the population into non-overlapping groups (i.e., strata) N1, N2, N3,... Ni, such that N1 + N2 + N Ni = N. Then do a simple random sample of f = n/N in each strata

Systematic Random Sampling Procedure:Number the units in the population from 1 to N; decide on the n (sample size) that you want or need;k = N/n = the interval size;randomly select an integer between 1 to k then take every k th unit

Cluster (Area) Random Sampling Objective: This strategy will help us to economize on our mileage Procedure: Divide population into clusters (usually along geographic boundaries); randomly sample clusters; measure all units within sampled clusters

Multi-Stage Sampling Various combinations of previously mentioned sampling methods (simple, stratified, systematic and cluster)

Block Diagram TEXT

Table TEXT Title A Title B Title C Title D Title E Title F

3-D Pie Chart TEXT

Marketing Diagram Title TEXT