© 2009 by The McGraw-Hill Companies, Inc. Research Methods in Psychology Observation

© 2009 by The McGraw-Hill Companies, Inc. Observational Research  Researchers cannot observe all of a person’s behavior all people’s behavior  Researchers can observe samples of individuals samples of behavior at particular times samples of different settings and conditions

© 2009 by The McGraw-Hill Companies, Inc. Observational Research  Goal of samples represent larger population of  behaviors  people  settings and conditions

© 2009 by The McGraw-Hill Companies, Inc. Observational Research  Example: In a typical week, how many hours of television do you watch?  What is the average number of hours for the class? Is this average representative of the number of hours of TV watched by  all students on campus?  all college students?  all people?

© 2009 by The McGraw-Hill Companies, Inc. Observational Research  Use data from a sample to represent the population “generalize” the findings from sample to population  External validity extent to which a study’s findings may be used to describe  people, settings, conditions beyond those used in the study

© 2009 by The McGraw-Hill Companies, Inc. Observational Research  Generalize findings sample must be representative of population  is sample similar to population?  do we know characteristics of entire population?  Psychology studies with college student samples are psychology students representative of larger population?

© 2009 by The McGraw-Hill Companies, Inc. Sampling Behavior  Extent to which observations may be generalized (external validity)‏ depends on how behavior is sampled  Two methods time sampling situation sampling  Goal: obtain representative sample of behavior

© 2009 by The McGraw-Hill Companies, Inc. Sampling Behavior, continued  Time Sampling choose time intervals for making observations  systematic  random don’t use time sampling for observing behavior during rare events (e.g., hurricane)‏  event sampling

© 2009 by The McGraw-Hill Companies, Inc. Sampling Behavior, continued  Situation Sampling choose different settings, circumstances, conditions for observations enhances external validity use subject sampling to observe some people within a situation

© 2009 by The McGraw-Hill Companies, Inc. Classification of Observational Methods  Categories based on intervention by researcher Observation without Intervention Observation with Intervention  Categories based on methods for recording behavior comprehensive record selected behaviors

© 2009 by The McGraw-Hill Companies, Inc. Observation without Intervention  Naturalistic Observation observation in natural (real-world) setting without attempt to intervene or change situation use when ethical considerations prevent experimental manipulation  Goals describe “normal” behavior, examine relationships among naturally occurring variables establish external validity of lab findings

© 2009 by The McGraw-Hill Companies, Inc. Observation with Intervention  Most psychological research involves intervention  Three methods in natural settings participant observation structured observation field experiment

© 2009 by The McGraw-Hill Companies, Inc. Observation with Intervention, continued  Participant observation observer is active participant in the natural setting he or she observes  undisguised: people know they’re being observed  disguised: people don’t know they’re being observed

© 2009 by The McGraw-Hill Companies, Inc. Observation with Intervention, continued  Problems with participant observation Reactivity  when people change their usual behavior because they’re being observed  disguised participant observation controls reactivity Observers lose objectivity or become too involved in situation Observers influence behavior of people they’re observing

© 2009 by The McGraw-Hill Companies, Inc. Observation with Intervention, continued  Structured observation set up (structure) specific situation in order to observe behavior used when behavior is difficult to observe as it naturally occurs researchers use confederates to structure situations problems: when observers don’t follow same procedures across observations

© 2009 by The McGraw-Hill Companies, Inc. Observation with Intervention, continued  Example of structured observation Simons and Levin (1998): “change blindness” Web site: go to “A subject in a real-world person change event”

© 2009 by The McGraw-Hill Companies, Inc. Observation with Intervention, continued  Field Experiment manipulate independent variable in natural setting and observe behavior (dependent variable)‏  two or more conditions to compare (IV)‏  often use confederates to create conditions  strive for control in natural setting

© 2009 by The McGraw-Hill Companies, Inc. Recording Behavior  Comprehensive record video, audio recordings  Select specific behaviors checklists, ratings  Method for recording behavior determines how results are measured, summarized, analyzed, reported

© 2009 by The McGraw-Hill Companies, Inc. Recording Behavior, continued  Qualitative Records Narrative records: complete reproduction of behavior (video, audio, field notes)‏ made during or soon after behavior occurs carefully train observers advantage: can review record often disadvantage: costly, time-consuming

© 2009 by The McGraw-Hill Companies, Inc. Recording Behavior, continued  Quantitative Records Selected behaviors Requires decision regarding how to measure behavior (e.g., frequency, duration)‏ checklists, electronic recording and tracking  Measurement Scales Four levels for quantifying behavior  nominal, ordinal, interval, ratio

© 2009 by The McGraw-Hill Companies, Inc. Measurement Scales  Nominal classify behaviors, events, and characteristics into categories checklist examples:  sex (male/female)‏  volunteer to help (yes/no)‏  grade (pass/fail)‏ lowest level of measurement

© 2009 by The McGraw-Hill Companies, Inc. Measurement Scales, continued  Ordinal rank-order behaviors and events 1st, 2nd, 3rd, etc. example: letter grades on a test (A, B, C, …)‏  Question: What is the average grade?  can’t compute averages using ordinal measurement

© 2009 by The McGraw-Hill Companies, Inc. Measurement Scales, continued  Interval scale allows researcher to specify distance between observations on a given dimension distance between points on scale is equal example: scores on a test big advantage: compute means and standard deviations  allows easy summary of groups question: does someone who scored 90 on a test know twice as much as someone who scored 45?

© 2009 by The McGraw-Hill Companies, Inc. Measurement Scales, continued  Ratio same as interval, but score of zero is meaningful example: time  someone who finished test in 50 min took twice as long as someone who finished in 25 min Most psychological measurement:  no meaningful score of “0”  we don’t recognize concepts such as “zero memory” or “zero intelligence”

© 2009 by The McGraw-Hill Companies, Inc. Measurement Scales, continued  Most psychological measurement interval scales are assumed example: rating scale of aggression not aggressivevery aggressive rating scales are treated as interval scales but are more accurately described as ordinal Is psychological distance between different points on the scale equal?

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data  Method for analysis depends on how data are recorded measurement scale  Two types of analysis qualitative quantitative

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued  Qualitative Analysis comprehensive, narrative records three steps  code data from narrative record  display the data  draw and verify conclusions

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued  Example of qualitative data analysis: teacher effectiveness based on videos of teachers and students (1)Develop coding scheme classify various teacher and student behaviors teacher: use of questions, examples, humor student: ask questions, boredom, note-taking

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued (2) Display data. example: Develop a sequence of effectiveness during classroom period in which effective teachers begin with anecdote then present an example then ask a question then insert humor, etc. Display this in diagram of nodes from one event to next

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued (3) Draw and verify conclusions. example: develop a theory that an effective teacher uses several different types of engagement throughout a class period

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued  Problems with Qualitative Analysis How to determine if theory is correct? Validity (truthfulness) of qualitative analyses often is questioned  The data (evidence) to develop theory often are the same data used to support the theory Qualitative analyses can be circular  Researchers’ biases can influence which data they examine to support their theory

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued  Quantitative Analysis selected behaviors method of data analysis depends on measurement scale used to record behavior

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued  Nominal summarize behavior using relative frequency  example: how many people in each category or percentage of people in a category  Interval or Ratio summarize behavior using central tendency (mean) and dispersion (standard deviation)‏

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued  Reliability refers to consistency Do two or more observers agree in their observations? (are they consistent with each other?)‏ interobserver reliability  example: suppose 2 observers rated “teacher effectiveness” on a 1–5 scale (not effective–very effective)‏

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued Suppose  Observer 1 rated Instructor Z with a “1”  Observer 2 rated Instructor Z with a “4” Question: Is Instructor Z effective? Can’t tell because of low interobserver reliability

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued  Factors that affect interobserver reliability characteristics of the observers  bored, tired, amount of experience  train observers and provide feedback clearly define events and behaviors to be observed  provide examples

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued  How to assess interobserver reliability Nominal scale: percentage agreement number of times 2 observers agree number of opportunities to agree X 100

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued interobserver reliability with nominal scale example: Is the teacher effective (yes/no)‏ Observer 1Observer 2 teacher 1yesyesagree teacher 2yesnodisagree teacher 3noyesdisagree teacher 4nonoagree Percent agreement:# of times agree = 2 =.50 X 100 = 50% # of opportunities 4 50% is low interobserver reliability (< 85%)‏

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued interobserver reliability with interval and ratio scales correlation coefficient Obs 1Obs 2Obs 1Obs 2 teacher 1 43teacher 1145 teacher 255teacher 1244 teacher 321teacher 1333 teacher 433teacher 1443 teacher 533teacher 1555 teacher 644teacher 1644 teacher 722teacher 1723 teacher 812teacher 1822 teacher 911teacher 1932 teacher 1033teacher 2055

© 2009 by The McGraw-Hill Companies, Inc. Analysis of Observational Data, continued  correlation between ratings for Observer 1 and Observer 2 =.88  acceptable reliability is correlation greater than.85

© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Observational Research  Problems in observational research influence of the observer on behavior observer bias

© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Observational Research, continued  Influence of the Observer Reactivity: people change their usual behavior when they know they’re being observed Researchers want to observe people’s usual behavior Demand characteristics: people pay attention to cues and information in the situation to guide their behavior

© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Observational Research, continued  Controlling reactivity conceal observer (ethics: privacy issues)‏ disguised participant observation (privacy)‏ use indirect (unobtrusive) observation Adapt participants to observer  habituation  desensitization  Reactivity is a potential problem in all psychological research

© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Observational Research, continued  Observer bias Observers often have expectations about behavior  example: expectations based on research hypotheses Expectations can lead observers to look at only particular behaviors Observer bias: systematic errors in observation that result from expectations  also called expectancy effects

© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Observational Research, continued  Observer bias potential problem in all research hard to eliminate observers must always be aware that they may be biased reduce bias by keeping observers “blind” to aspects of the study:  reasons for observations  goals of the study  hypotheses