1. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Chapter 7 Using Nonexperimental Research

2. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Developing Behavioral Categories A behavioral category includes the general and specific classes of behavior to be observed Categories must be operationally defined Developing behavioral categories may be easy or challenging Behavioral categories must be clearly defined to avoid confusion Begin with clear goals for research Clearly define all hypotheses Keep categories as simple as possible Avoid temptation to accomplish too much in one study

3. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Quantifying Behavior in Observational Research Frequency Method Record the frequency with which a behavior occurs within a time period Duration Method Record how long a behavior lasts Intervals Method Divide the observation period into several discrete time intervals (e.g., ten 2-minute intervals), and record whether a behavior occurs within each interval

4. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Coping With Complexity in Observational Research Time Sampling Scan subjects for a specific period (e.g., 30 seconds), and then record your observations during the next period Individual Sampling Select a subject and observe behavior for a given period (e.g., 30 seconds), and then shift to another subject and repeat observations

5. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Event Sampling Select one behavior for observation and record all instances of that behavior It is best if one behavior can be specified as more important than others Recording Use a recording device to make a record of behavior for later review

6. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Evaluating Interrater Reliability You must establish reliability of observations from multiple observers (interrater reliability) Methods for evaluating interrater reliability Percent agreement Simplest method Percent agreement should be around 70% Percent agreement may underestimate agreement Cohen’s Kappa Popular method Allows you to determine if agreement observed is due to chance

7. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Pearson Product-Moment Correlation Correlate ratings of multiple observers with Pearson r Simple and easy method to evaluate interrater reliability Two sets of scores may correlate highly, but may still differ markedly Intraclass Correlation (ICC) Extension of Analysis of Variance logic to interrater reliability A powerful and flexible tool for evaluating interrater reliability

8. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Interrater Reliability: Using Cohen’s Kappa Tabulate frequencies of interrater agreement and disagreement in a CONFUSION MATRIX Determine the proportion of actual agreement by summing the values along the diagonal of the confusion matrix and dividing by the total number of observations Find the proportion of expected agreement by multiplying corresponding row and column totals and dividing by the number of observations squared Enter resulting numbers in the formula for Cohen’s Kappa A Cohen’s Kappa of.70 or more indicates acceptable interrater reliability

9. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Nonexperimental Approaches to Data Collection Naturalistic Observation Unobtrusive observations of subjects’ naturally occurring behavior are made Ethnography The researcher becomes immersed in the behavioral or social system being studied. May be conducted as a participant or non-participant observation study Sociometry You identify and measure interpersonal relationships within a group

10. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Case History You observe and report on a single case Archival Research You use existing records (e.g., police records) as your source of data Content Analysis You analyze spoken or written records for the occurrence of specific categories of events (e.g., a word or phrase) Both RECORDING and CONTEXT UNITS are evaluated

11. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Issues to Be Considered in Ethnography Observing as a participant or non-participant Gaining access to a field setting Gaining entry into the group Becoming invisible Making observations and recording data Analyzing ethnographic data

12. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Content Analysis: Defining Characteristics Used to analyze a written or spoken record for occurrence of specific behaviors or events Archival sources often used as sources for data Appears simple, but may be complex Should be used within a clearly developed study, including hypotheses to be tested Response categories must be clearly defined A method for quantifying behavior must be defined

13. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Performing a Content Analysis Clearly defined response categories are essential Two units of analysis Recording unit: Element of the material you are going to record (e.g., instances of a certain word) Context unit: Context within which material analyzed appears Observers doing content analysis must be blind so that bias will not enter the analysis Materials to be analyzed should be chosen carefully to increase generality Cannot be used to establish causal connections among variables

14. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Factors to Include When Meta-Analyzing Literature Full reference citation Names and addresses of authors Sex of experimenter Sex of subjects used in each experiment Characteristics of subject sample (e.g., how obtained, number) Task required of subjects and other details about the dependent variable

15. © 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Design of the study (including any unusual features) Control groups and procedures included to reduce confoundings Results from statistical tests that bear directly on the issue being considered in the meta-analysis (effect sizes, values of inferential statistics, p values)