Data Collection Methods Chapter 11 Data Collection Methods

Observational research: Observing the behavior of individuals Watching people and systematically recording their behavior. Can be used to collect data in experimental and non-experimental research (text focuses on non-experimental). It may be difficult or impossible to observe some behaviors we are interested in.

Observational research Observing behavior from the outside: Naturalistic observation Involves making systematic observations of behavior in the environment where it occurs naturally. Used to study behavior as it naturally occurs and in a way that is as unobtrusive as possible. Well suited for studying behaviors that we fear would be altered or not occur at all if the participants knew they were being observed (reactivity effect). High external validity. Can be time-consuming and inconvenient.

Observational research Observing behavior from the inside: Participant observation The researchers enter the world of the people they are interested in studying and maintain field notes chronicling their observations. Usually provides rich narratives about the experiences of the observed and the observer. Advantages: Introduces us to the world of others. High external validity. The researcher is able to experience the same environmental conditions as the participants. Can collect information on factors that may not be overtly observable.

Observational research Observing behavior from the inside: Participant observation Subjectivity is a major advantage of participant observation and it is also a major disadvantage. Researchers must be able to maintain/regain their objectivity and not misrepresent or bias their observations as a result of their experiences. Disadvantages: Time consuming and potentially dangerous. May be difficult to gain entrance into certain groups. Ethical issues (e.g.. need to maintain anonymity of your informants).

Observational research Observing behavior in a controlled setting: Laboratory observation When we observe behavior in the laboratory, we can create the conditions necessary to make the behavior occur. Trade-off between external validity and level of control. May lack external validity.

Observational research Advantages and disadvantages of observational research In all types of observational research it is important to form clear definitions of the behaviors you are interested in observing. Wherever research is conducted, the researcher must be accountable for any harm to the participants. Steps must be taken to guard the rights of those involved. Observational research is often time-consuming and requires observers who are trained to carefully record behavioral observations.

Survey research: Asking people questions about their behavior Surveys Used to measure people’s opinions and attitudes, variables that may be difficult or impossible to observe directly. Defining your research question From start to finish, your research question or hypothesis must guide you. Need to be certain about your purpose.

Survey research How will you ask your questions? Survey – refers to the action of collecting information. Questionnaire – a list of questions that are asked when you are collecting information. Need to determine the kinds of information you need and the best method of obtaining that information. Also need to consider who your respondents are. Time, money, literacy, and respondent honesty are all factors that need to be considered when making the choice.

Survey research Interviews Interviewing can be very expensive and time consuming, and it requires trained interviewers. Allows rapport to be built, which can be an advantage and a disadvantage. Includes: Face-to-face interviews Telephone interviews

Interviews Face-to-face interview Generally, if the population you want to study is available at a particular location and your questions don't take more than about five minutes, then the simplest approach is to go to where your group is and interview them in person. Do not need to have respondents complete an informed consent form before they answer your questions, but still need to consider the ethics of consent.

Interviews Face-to-face interview Advantages and disadvantages of face-to-face interviews Advantages Gather information directly from the people you are interested in researching. Can explore complex issues that do not lend themselves to multiple-choice answers (e.g.. probing questions and open-ended questions). [Note: it is very important that the probes be neutral so as not to bias responses.]

Interviews Face-to-face interview Advantages and disadvantages of face-to-face interviews Disadvantages Can be very time consuming. Participant discomfort in discussing personal or embarrassing topics. Social desirability - the tendency for people to respond in a manner that makes them appear better than they are. Labor intensive. Practical limitations in terms of the number of participants.

Interviews Telephone interview Best choice if your research question requires interviewing a large number of respondents who are spread over a large geographical area. Generally shorter than face-to-face interviews. Data are often coded and entered into a computer directly by the telephone interviewer.

Interviews Telephone interview Advantages and disadvantages of telephone interviews Advantage: Don’t have to be in the same location as the respondent. Disadvantages: Selection bias as a results of individuals screening calls. Time consuming. Require trained interviewers.

Survey research Types of questionnaires: Self-administered questionnaires Group-administered questionnaires Mail-out questionnaires Internet questionnaires

Questionnaires Self-administered questionnaires Survey questions that are read and answered by the respondent with little or no direct contact with the researcher. Advantages: Cheap Fast Anonymous Requires literacy and for the questionnaire to be well written.

Questionnaires Group-administered questionnaires If it is possible to gather your respondents into groups (e.g.. students in classrooms), then group administration is probably your best option. Need someone to distribute and collect the questionnaire. Advantages Inexpensive Fast High response rates Someone is present while the participants complete the questionnaire. This administrator can give verbal instructions and answer questions.

Questionnaires Mail-out questionnaires Quick and economical method for distributing your questionnaire to a large number of people spread over a large area . Gives respondents a feeling of confidentiality and anonymity. Main problem: poor response rates. This results in increased costs and potentially poor validity as the respondents may not be representative of the population. You do not control who completes the questionnaire.

Questionnaires Internet questionnaires Provides respondents with a strong feeling of anonymity. Allows you to target special groups. Many of the same problems as other survey research: People who don’t use the Internet will not be included in an Internet survey. Social desirability and other kinds of response bias. Poor response rate. Internet research has special problems such as multiple submissions from the same respondents.

Survey research General guidelines for writing survey questions Wording of your questions is important. Ensure that your questions are asking what you intend them to ask by pre-testing your items. Do a pilot study to detect other problems before beginning the full study.

Writing Survey Questions Guidelines: Keep your questions short and simple. This helps ensure respondents clearly understand your questions. Avoid using ‘and’ in your questions. You should measure no more than one dimension with each question. Do not use biased wording in your questions. Be sure you avoid using double negatives. Can be confusing for respondents.

Type of questions Open-ended questions Items that simply have a blank space for the response. Best choice if you are not sure how people will answer a question or if you are looking for diverse responses. Disadvantage is that they may be difficult to analyze. Can be time consuming and costly.

Type of questions Forced choice questions Items that include response categories. Easier to analyze and easier to answer. Must be sure that the response choices include all the possible responses people can make. You may include an “other” category – be sure to leave a blank for people to enter their response. Need to provide clear instructions that only one response may be selected.

General rules for self-administered questionnaires Very important that it looks professional. Should be easy to read and organized in a way that is easy to follow. Mailed questionnaires should include a cover letter with sufficient information for participants to decide to whether or not to participate (like informed consent). Use clear organization so that your respondent does not get lost filling it out (e.g.. try to organize the questions by topic).

Observing group behavior: The focus group Could be described as a face-to-face interview of a small group of people in which participants can interact and generate ideas from one another. Often used to measure attitudes, opinions, or self-reported behavior of select groups. Advantage of revealing information that may not surface in face-to-face interviews. Dynamic nature of focus groups can lead to problems (e.g.. a few forceful members dominate the discussion). Usually recorded so the content of the discussion can be analyzed later using qualitative techniques.