1. Research Methods Psych 402 LECTURE 9 – Sampling VHS 514G2 (#14 – sampling)

2. Efficacy & Effectiveness studies Efficacy studies –Active vs Placebo condition where there is no change in behavior expected (Placebo effect - spontaneous recovery - is around 30 % for numerous conditions) Effectiveness study: –Active vs current best treatment e.g., new drug vs Prozac in depression More ethical

3. Sampling

4. What is Sampling? “Sampling is the process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen.” - William Trochim, 2002

5. Sample Characteristics At issue: representative of population of interest?

7. Sampling Key Terms Population - complete set of individuals having some common characteristic – e.g., Australians, Buffalo Bills fan Sampling frame – subset of the population from which the sample is actually drawn – e.g., White pages, game attendees Sample – the set of people actually studied (i.e., selected from the sampling frame) – e.g., Every 1000 th person in the white pages, every 10th season ticket holder (which would not be a representative sample of this population)

8. Sampling Techniques Probability sampling - each member of population has a specific probability of being chosen. Non-probability sampling - arbitrary sample, often chosen with a purpose, that is not representative of the population

9. Probability Sampling Techniques Random Sampling - everyone in population has an equal chance of being selected.

10. Probability Sampling Techniques Stratified Random Sampling - population divided into strata, then random sampling from within each stratum

11. Probability Sampling Techniques Cluster Sampling - identify ‘clusters’ of individuals, randomly select among cluster, then sample exhaustively inside a cluster

12. Video Against All Odds, VH 514G part 2 –Episode 14 - Surveys and Sampling

13. Nonprobability Sampling Self-selected Haphazard –(e.g., canvas laundromat, beach, door-to-door, by phone) Convenience –(e.g., clients, nearby clinic, psych students) Systematic –(e.g., 1 st 50 people in clinic) Restricted –(e.g., SAT MS>NJ - 3 v 65%) Purposeful

14. Non-Probability Purposeful Sampling Quota Sampling – meet a quota –e.g., 50% psych, 30% economics, 20% law students Modal – typical registered voter Expert – ask only experts for a topic Heterogeneity – seeking as many different individuals or type of opinions as possible Snowball Sampling - ask each respondent if they know someone else suitable for survey –e.g., studying drug-users.

15. Representativeness of Sample depends on: adequacy of sampling frame –accidental stratification with strata missing selection strategy adequacy of sample size response rate –percent & representativeness of those who complete Small, good sample is better than large, poor sample (Gallup’s 50,000 vs Literary Digest’s 3 million poll of 1936 Roosevelt election)

16. External Validity

17. Threats to External Validity

18. Sampling Example: Sher Hite ‘American Sexology’

19. Hite Report: Male-Female Relations 127-item questionnaire about marriage and relations between men and women 4500 USA women, 14 to 85 years

20. Some of Hite’s findings.... 70% married for 5 years had one or more affairs... 87% had a closer female friend than husband 98% wanted “basic changes” to love relationships 84% were emotionally unsatisfied 13% were still in love with husband (after 2+y of marriage) “She goes in with prejudice & comes out with a statistic.”

21. Methodology 100,000 questionnaires to a variety of women’s groups - feminist organisations, church groups, garden clubs, etc. 4,500 replied (4.5% return rate)

22. “We get pretty nervous if respondents in our survey go under 70%. Respondents to surveys differ from nonrespondents in one important way: they go to the trouble of filling out what in this case was a very long, complicated, and personal questionnaire.” Regina Herzog, Michigan Institute for Social Research Sample Representative? - selection bias - nonresponse bias

23. Hite Revised by others When representative sample studied by others, rate of female infidelity fell to single digits, etc.

24. Sources of Error Sampling Error (non-representative sampling frame) Nonsampling Error non-response bias (those who respond doesn’t equal those who don’t) inaccurate response –unreliable measurements –misunderstanding of instructions –social desirability or opposite –researcher expectancy

25. Minimizing Error Sampling - take pains to ensure sampling frame is representative Non-sampling – well-designed measures – trained interviewers – call backs (to eliminate nonresponse) – data checks

26. Research Methods Psych 402 LECTURE 10 – Library Research & Hypothesis Testing VHS 514J (#20)

27. Library Query Assignment Are motor skills impaired when you’re anxious? Are driving skills impaired by marijuana use? Does smoking pot make you less aggressive? Does drinking alcohol make you more aggressive? Are teenagers who smoke more antisocial than teenagers who don’t? Are children of alcoholics more likely to divorce than other groups? Are children of alcoholics more likely to develop mental illness? Does divorce alter a child’s self-esteem? Does teenage sex alter a girl’s self-esteem? Does teenage sex alter a boy’s self-esteem? Does watching violent TV make boys more aggressive? Does watching violent TV make girls more aggressive? Does watching TV impact school performance? Is episodic memory impaired by drinking alcohol? Is semantic memory impaired by drinking alcohol? Are photographic memories true records of events? Is eyewitness testimony reliable? Are women better at perceiving emotions than men? ….

29. APA Citation Format: Journal or Book Authors (Year). Article title. Journal Title, Volume Number, pages. Roosevelt, F.D., and Truman, H. (1945). Evolutionary strategies in politics. American Journal of Research, 35, 124- 138. Authors (Year). Book Title. City published: Publisher. Arnheim, R. (1971). Art and visual perception. Berkeley: University of California Press. –Last Name, F. M., Last Name, F.M, and Last Name, F.

30. APA Reference Format: Book Authors (Year). Book Title. City published: Publisher. Arnheim, R. (1971). Art and visual perception. Berkeley: University of California Press.

31. Research Methods Psych 402 Hypothesis Testing

32. What is a hypothesis? 1. It is a clear statement about the state of the world, a prediction between variables and/or conditions. It is not a question 2. Identifies specific relationships between variables 3. Based in theory or existing knowledge 4. Concise 5. Testable 6. Readily understood by others

33. Example Hypotheses “ Older people drive less aggressively than younger people” “The number and severity of driving infractions is negatively correlated with age, from 16 to 80 years of age”

34. Hypothesis testing Research hypothesis - “It is predicted that older drivers will have fewer driving infractions than younger drivers” Test the null hypothesis: i.e., that there is no relationship – can H 0 be rejected or not?

35. Reject the null hypothesis (accept H 1 ) or fail to reject the null hypothesis (accept H 0 ) Probability for rejecting the null - conventionally set at p <.05 (significance level)

37. VH 514 J #20

38. Designs The simplest of all experimental designs is the two-group posttest-only randomized experiment. In design notation, it has two lines -- one for each group -- with an R at the beginning of each line to indicate that the groups were randomly assigned. One group gets the treatment or program (the X) and the other group is the comparison group and doesn't get the program (note that this you could alternatively have the comparison group receive the standard or typical treatment, in which case this study would be a relative comparison).

40. Research Methods Psych 402 LECTURE 11 – Unobtrusive Measures VHS Natl Geographic Apes

42. Three ways to “observe” people Directly –Observational –Survey (sometimes indirect) Indirectly [unobtrusive measures] –Use Traces & other physical evidence –Archival (may be primary, direct source)

43. Example: Which game was more exciting? Today’s Red Sox/Yankees or the Championship Series game Birds vs Cards: 7-3Birds vs Cats 6-0

44. Example: Which game was more exciting? Bills or Patriots game Sound volume (overall or maximum) # viewers # lead changes Self-report # fans left early Score # extra innings Importance of game Food sales

45. Unobtrusive Measures Use Traces Routine Records Documents Products indirect measures

46. Use Traces – any changes in environment due to human actions Erosion: measurement of selective wear on material –Broken windows in public bldgs – community pride –Tile replacement in front of museum exhibit – public’s interest Accretion: evidence of some deposit of materials –Ticker tape volume – parade popularity –Dust on library books – recency and famount of use

47. Advantages of physical traces Actual behavior can be measured Physical properties are quantifiable Cumulative – often reflects long process Nonreactive and unobtrusive (usually)

48. Archival Research Not all research means collecting new data –Advantages – non-reactive, cheap to collect (not so to interpret), usually a high quality source, continuous –Content analysis – inferences about specific characteristics of messages –Multi-method approach (converging datasets) Possible problems –Selective deposit (bias in production) –Selective survival (discontinuous, outdated) –Possible errors in record keeping, storage –Perceiving spurious relationships

49. Examples of Routine Records

50. Observational Research Decide on: –Artificiality of Setting –Any interaction with Observed –Awareness of the Observed

51. How to perform Observational Research Structured –Recording specific features –Determined prior to observation –Partial record of occasion’s behaviors –Checklist or written protocol –Observed behavior predictable –ALLOWS testing of hypotheses and estimation of observer reliability Unstructured –All that is relevant –Determined at the time of observation –More complete –Open format –Unpredictable behaviors captured –GENERAL description only, with no estimation of reliability

52. Checklist for Structured Observation

54. Conditions for the Use of Observation –routine records & here Data must be assessable to observation (e.g., practices, skills, not motivation or attitudes, etc) Behavior must be repetitive, frequent, or otherwise predictable Event must cover a reasonably short time span (time and cost) – one may sample phases in longer behaviors

55. Advantages of Observation Involves direct experience, not 2 nd -hand Active cooperation of subject not required (may even be unaware or unbothered) May provide greater detail of info compared to other methods) More valid than direct questioning Permits data collection with queries impossible Can combined with other data collection techniques for quality assurance & verification

56. Disdvantages of Observation May lead to reactivity of the observed party Relies heavily on personal interpretation (biases, degraded objectivity, calibration shifts, focus on exotic data) May not be realistic for large population May not be representative of entire population Often unsystematic, no standard operating procedures across all settings, etc.

58. Cognitive Development “Never hug and kiss them, never let them sit on your lap. If you must, kiss them once on the forehead when they say good night. Shake hands with them in the morning. Give them a pat on the head if they have made an extraordinarily good job of a difficult task. Try it out. In a week’s time you will find how easy it is to be perfectly objective with your child and at the same time kindly. You will be utterly ashamed at the mawkish, sentimental way you have been handling it.” — John Watson “Psychological Care of Infant and Child”

59. Harlow’s Test of Freudian Thought: Infant loves the breast, not the mother (sustenance vs bodily contact) Harlow’s Surrogate Mother Experiments –Monkeys preferred contact with the comfortable cloth mother (e.g., spent 23 out of 24 hours on terricloth mother.

60. Importance of Attachment Monkeys raised by artificial mothers were terror-stricken when placed in strange situations without their surrogate mothers

61. Natl Geographic video