Different Approaches for Data Collection/Data Analysis Experiment Survey Quasi-Experiment Secondary Data Analysis Content Analysis Historical Comparative Methods (Archival Analysis ) Participant Observation Ethnography Individual Qualitative Interview Textual Analysis Focus Group Discussion Del-Phi Method

Secondary Data Analysis A type of research in which data collected by others are reanalyzed. Primary Data Analysis Original analysis of the data collected in a study. Meta-Analysis “Analysis of analysis” Quantitative procedure for summarizing or integrating the findings obtained from a literature. Uses the results of individual research projects on the same topic as data points for a statistical study of the topic.

Secondary Data Analysis Advantage Saves cost for data collection. Saves time required for data collection. May obtain data otherwise impossible. Disadvantage Data collected may not be suitable for the researcher’s purposes. (Validity) All necessary data may not be available in existing data. Requires time to search for the data set. Original data set may not be accurate. Existing data sets may contain more variables than what the researchers need. Need time to sort out the data.

Content Analysis General usage of the term Refers to any of several research techniques used to analyze the content of written, spoken, or pictorial communication such as books, newspapers, television programs, or interview transcripts. Specific usage of the term Refers to a quantitative technique that attempts to quantify the meaning of the content of the written, spoken or pictorial communication.

Content Analysis 1. Determine “Unit of Analysis” 2. Determine “Indicators” 3. “Code” the data Example Research Question: What values and beliefs were expressed by advocacy coalition about Social Security? Unit of Analysis: Congressional testimony on Social Security reform from 1983 to Indicators: Values---Statement on what they like. Beliefs---Statement on what things should be.

Content Analysis 1. Issue of “Sampling” e.g. Randomly sample 20 congressional testimony each month. 2. Issue of “Coding” Coding Sheet Intercoder Reliability

Exercise: Content Analysis of Course Syllabus Content: PA410U Syllabus Coding: Civic Capacity (CCI ) Initiative Matrix

Historical Comparative Methods (Archival Analysis) Focus on identifying historical trends or patterns. Focus on comparing the historical trends or patterns. Quantitative approach: Secondary Data analysis. Content analysis of archival data. Qualitative approach: Oral history. Case study.

Qualitative Research Methods Examples: Participant Observation Ethnography Individual Qualitative Interview Textual Analysis Focus Group Discussion Del-Phi Method

Key features of Qualitative Research 1. Collection primarily of qualitative rather than quantitative data Qualitative methods emphasize observations about natural behavior and artifacts that capture social life as it is experienced by the participants rather than the numerical representations of the categories predetermined by the researcher. 2. Exploratory research question. Qualitative researchers typically begin their projects seeking to discover what people think and how they act, and why, in some social setting. 3. Inductive reasoning (Reasoning that moves from more specific kinds of statement to more general ones) Only after immersing themselves to many observations, do qualitative researchers try to develop general principles to account their observations.

Key features of Qualitative Research 4. A focus on human subjectivity. Qualitative methods emphasize the meanings that participants attach to events and that people give to their lives. 5. Reflexive research design. In the qualitative methods, the research design may need to be reconsidered or modified in response to new developments, or to changes in some other component as research progresses. 6. Sensitivity to the subjective role of the researcher. Qualitative researchers should be sensitive to the role they play in the process of data collection. “Researcher as an instrument”

Ways to collect qualitative data 1. Participant Observation 2. Ethnography 3. Individual Interview a) Semi-structured interview b) Unstructured interview 4. Textual Analysis 5. Focus Group Discussion 6. Del-Phi Method

Participant Observation Sometimes referred to as “fieldwork” in anthropology. A method in which natural social processes are studied as they happen. Researcher takes a role of “participant”

Ways to collect qualitative data 1. Participant Observation 2. Ethnography 3. Individual Interviews

Individual Interviews Semi-structured interview Interviews with an interview guide containing primarily open-ended questions that can be modified for each interview. Unstructured interview The interviewer starts with only a general sense of the topics to be discussed and creates questions as the interaction proceeds.

Interview Guide A list of topics to cover and the order in which to cover them that can be used to guide less structured interviews. The questions must be judged its appropriateness for each interview, and may be re-ordered, and re- worded if necessary.

Types of Questions Open-ended questions vs. Close-ended question Open-ended “How would you describe the performance of the new Japanese Prime Minister”? Closed-ended “Do you think the new Japanese Prime Minister is doing a good job?” Probes Questions that allow the interviewee to further expand on their earlier comments. “Can you elaborate on what you mean by...” “Why is that so?”

Ways to collect qualitative data 1. Participant Observation 2. Ethnography 3. Individual Interview 4. Textual Analysis 5. Focus Group Discussion 6. Del-Phi Method

Focus Groups A type of group interview. Focuses on group interaction on a topic selected by the researcher. Ideally participants. The interaction is directed by a moderator who asks questions and keeps the discussion on the topic. Reference: Dr. David Morgan (Institute on Aging, PSU) “Focus groups as qualitative research”(1995). Sage. Annual Review of Sociology 22, p (1996)

Ways to collect qualitative data 1. Participant Observation 2. Ethnography 3. Individual Interview 4. Textual Analysis 5. Focus Group Discussion 6. Del-Phi Method

Different Purposes for Research Pure Research : Discovery of knowledge for the sake of knowledge. Applied Research: Conducted with some practical purposes in mind. Exploratory Research: Conducted in order to explore the phenomena. Explanatory Research : Conducted in order to explain the phenomena. Evaluation Research: Conducted in order to develop and “evaluate” programs.

Evaluation research Research specifically designed to assess the impact of a specific program, policy or legal change. Often the focus of an evaluation is whether the program, policy, or law has succeeded in effecting intentional or planned change.

Developing and Evaluating a Program Research Program Research Program Needs assessment Design a program Implement a program Process Evaluation Outcome Evaluation Redesign a program Or Terminate a program

Types of Evaluation 1. Needs Assessment An Analysis of whether a problem exists, its severity, and an estimate of essential services. 2. Process Evaluation/Formative Analysis Research that monitors a program or policy to evaluate the process of implementation of the program. Provides the program staff with evaluation of the current program and suggestions for improving the program design and service delivery.

Types of Evaluation 3. Outcome evaluation (Impact analysis, summative analysis) Research that is designed to “sum-up” the effects of program, policy, or law in accomplishing the goal or intent of the program, policy or law. 4. Cost-Benefit Analysis Research that compares a program’s costs to its benefits. A study designed to weigh all expenses of a program (its costs) against the monetary estimates of the program’s benefits (even putting dollar values on intangible benefits.)

Selecting a study design Quasi-Experiment An experiment design that is missing one or more aspects of the classic controlled experiment. Panel Study (Longitudinal Study) A study design in which data are collected about one sample at least two times and all variables are measured, not controlled by the researcher.

Selecting a study design Cross-sectional Study A study design in which data are collected once for all the variables of interest using one sample. Case Study A research strategy that focuses on one case (an individual, a group, an organization, and so on) with in its social context at one point in time, even if that one time spans months or years.

Summary Group Exercise