Field and Quasi-experiments

Review of main effects/interactions sheets

Mediation/moderation Results indicated that team autonomy was both directly and indirectly positively associated with company performance. Furthermore, organizational commitment partially explained the relationship between team autonomy and company performance. Women in the perfectionism condition and scoring high on body dissatisfaction exhibited the highest levels of eating disorder symptoms. The results reveal that mimetic and particular normative pressures significantly influence work–life practices, while coercive and global normative pressures appear not to do so. The perception of certain negative consequences related to the implementation of work–life practices inhibits this responsiveness.  Young people who tend to act rashly in response to extremely positive moods showed higher enhancement and coping motives, which in turn were positively related to gambling problems. According to the Justification-Suppression model of prejudice (Crandall & Eshleman, 2003), people covertly express prejudice through attitudinal ambivalence toward outgroups that is based on beliefs. We predicted that this would only be the case for people who are more externally, but not more internally, motivated to control their prejudice. 

Mediation/moderation

Analyzing moderators Case 1: Cat IV, Cat Mod: 2 x 2 ANOVA Case 2: Cont IV, Cat Mod: Could do correlations sep and compare BUT Better to do regression and compare unstandardized Betas Or SEM software

More moderators Case 3: Cat IV, Cont Mod: Case 4: Both Cont: Figure out how level of moderator affects IV-DV relationship If linear, do hierarchical regression, showing that XZ adds to the effects of X and Z on Y Case 4: Both Cont: Use Case 2 if step function or Use Case 3 if linear, quadratic How do you know which one is the moderator vs. IV?

Analyzing mediators Causal steps (Baron & Kenny) Limitations: IV related to M M = i + aX + e IV related to DV Y = i + c1X + e M and IV related to DV Y = i + c2X + bM + e C1 greater than C2 (look at size and sig) Limitations: Not good for multilevel, probit, logistic, survival Need to test for whether C1 > C2 (Sobel test) Low power esp when IV and DV aren’t related Overestimates effect of IV on DV if error in M

Sobel test Used to test whether C1 > C2 Good for sample sizes of 50+ with 1 M Or 100-200 for >1 M

Alternatives to B & K Use multiple measures of M and use SEM Distribution of the product PRODCLIN Better Type I error rates, higher power Computer-intensive methods Aka resampling Fewer assumptions

Assumptions of mediational tests Residuals are independent in equations 2 and 3 No XM interaction in equation 3 Direction is correct (DV doesn’t cause M) Measurement is perfect, esp. in M No unmeasured variables that cause X, Y, or M IV related to M M = i + aX + e IV related to DV Y = i + c1X + e M and IV related to DV Y = i + c2X + bM + e

Other mediation issues Complete vs. partial mediation Use hierarchical regression Test C2 significance Inconsistent mediation Multilevel mediation Can increase T1 error if you ignore Categorical DV mediation Use logistic or probit regression Multiple mediators

Longitudinal mediation Moderated mediation Mediated moderation Mediated baseline by treatment moderation

Meds and Mods How do you know which variable is the mediator? How can a moderator lead you to a mediator? How can a mediator lead you to a moderator? Can a variable be both a mediator and a moderator at the same time? Are there variables that are always going to be one or the other? Does a mediator have to be correlational? Remember that one study does not a mediator/moderator make

Field experiments NOT quasi-experiments Historical shifts NJ Negative Income Tax Experiment (hsty) What are potential problems with field experiments? How can field experiments be encouraged?

ITT vs. TOT What are they, and which should you care about? Vietnam draft example Problem of comparing those who served vs. not 35.4% of draft eligible served 19.4% not eligible served So 15.9% due to random assignment 2.04% draft elig vs. 1.95% not died So (2.04-1.95)/15.9 = .56%

Other field exp problems Attrition—how can this be decreased? Nested designs—how can these be dealt with? How can you improve power in nested designs? Intraclass correlations Ethical issues?

Why use quasi-experiments? When are they better?

Threats to internal validity Selection History Maturation Testing Instrumentation Mortality Regression Social interaction threats

Example #1 You want to study the effects of 24 hour visitation in the dorms of a college campus on grades. One all-female dorm voted for 24 hour visitation while another did not. You compare the grades of people living in each of these dorms the semester before and the semester after the visitation change. What kind of study have you done? What are some possible threats to internal validity? How could you correct them?

Example #2   The following graph represents the number of suicides per month in city high schools before and after a suicide intervention program that occurred in May 1996. What kind of study is this? Interpret the results. Was the program effective? What are the possible threats to internal validity?

Example #3 A researcher studying developmental changes in intelligence conducted the following cross-sectional study. The Wechsler Adult Intelligence Scale was given to 20 people at each of the following ages: 20, 30, 40, 50, 60, and 70. These are the results: Age IQ 20 120 30 110 40 100 50 95 60 93 70 89   From these results the researcher concluded that intelligence peaks in the early years and declines steadily thereafter. Do you agree? Why or why not?

Nonequivalent Control Group Design What is it? Examples. Figures 10-3 to 10-7 What makes a good control group? Focal local Matching Propensity score analysis Advantages and disadvantages Threats to internal validity

Interrupted time series What is it? Examples Advantages and disadvantages Threats to internal validity ABAB designs (and ABA, ABACBCBC ,etc.) Pooled time series/multilevel models

Regression Discontinuity Design What is it? Example Advantages and disadvantages Threats to internal validity Fuzzy RDD Figures 10-10

Pattern matching What is it? Example Advantages and disadvantages Threats to internal validity

Other quasi-exp designs Proxy pretest Separate pre-post samples Double pretest design Switching replications Nonequivalent dependent variables Regression point displacement

How else can you reduce threats to IV? Give examples: By argument By measurement or observation By design By analysis By preventative action

How to expand a basic design Across time Across programs Across measurements Across groups How do Trochim & Donnelly suggest you build a design?

Good design elements Theory-grounded Situational Feasible Redundant Efficient

Other design issues Why is theory important? How objective is research and how important is objectivity? How can you know that the treatment was delivered effectively?

1. An investigator attempted to ascertain the effects of hunger on aggression in cats. She took ten cats, kept them in individual cages, and put them on a food deprivation schedule such that at the end of two weeks the cats weighted 80 percent of their normal body weight. She then puts the cats in pairs for 15 minutes and watched to see if aggression or fighting would occur. In all cases, the cats showed the threat posture, and in most cases fighting occurred. The investigator concluded that hunger increases aggression in cats. 2. A psychologist examined the cause of failure among college students. She took a group of former students who had flunked out and a group of students who had received good grades. She gave both groups a self-esteem test and found that the group that flunked out scored lower on the test than did the group that received good grades. She concluded that low self-esteem is one of the causes of college failure and suggested further that a person with low self-esteem probably expects to fail and exhibits defeatist behavior in college, which eventually leads to failure. 3. An experimenter took 20 subjects who said they believed in astrology, gave them their horoscopes for the previous day, and asked them how accurate the horoscopes had been in predicting the previous day’s occurrences. The participants indicated their opinion on a 6- point scale that ranged from extremely accurate to extremely inaccurate. All 20 participants indicated that the horoscope was accurate to some degree. The experimenter concluded that horoscopes are accurate.

A recent newspaper article announced, “Candy Cigarettes Influence Kids” and went on to report that children who buy candy cigarettes are much more likely to smoke later on that are children who don’t buy them. The survey indicated that seventh-grade students who had purchased candy cigarettes at least twice were far more likely to have tried real cigarettes than were students who had not bought candy cigarettes. The results were so clear that school officials wanted to ban the sale of these candy cigarettes because, they contended, the candy cigarettes cause smoking in young children. The survey also indicated that in families where at least one parent smoked the children were much more likely to buy candy cigarettes and to have tried smoking on several occasions. Was the reaction by the school officials justified?

An experimenter interested in behavior modification decided to develop a new cure for fingernail biting. The cure she devised was a variant of cognitive modification in which the client was supposed to think about unpleasant events each time he or she bit a fingernail. Specifically, the experimenter had college-aged participants who bit their nails come to her office for a training session. The participants were instructed to think about the horrible mutilation of their fingers each time they chewed on a fingernail. The psychologist examined the nails of her participants prior to the onset of treatment and again three weeks later. The results showed that 87% of the participants had better fingernails at the posttest. The researcher concluded that the experiment was a success. Do you agree with this conclusion? Give three reasons why we shouldn’t be convinced.

A 2 x 3 factorial design was used to evaluate the effect of an experimental drug (Remoh) on the treatment of schizophrenia. Two patient classifications were used: 1) new admissions to a particular mental hospital; and 2) patients who had been institutionalized for at least two years at that hospital. Both groups had not been hospitalized previously. Patients received one of three levels of Remoh--3 g/day, 6 g/day, or 9g/day. Participants from each patient group were randomly assigned to one of the three dosage levels. There were 20 patients in each of the six groups. In addition to administering the drug, the experimenters also rated each patient each week on the presence or absence of schizophrenic symptoms. After two months, it was found that very few (10%) of the long-term patients in each group had improved, regardless of dosage level. But approximately half of the new patients had improved in each of the three dosage level groups. The researchers concluded that 1) Remoh is effective only for new arrivals and not for chronic cases, and 2) a dosage of 3 g per day is sufficient to maximize the effectiveness of the drug.  

I am interested in the effect of a nationwide law allowing female frontal nudity to be shown on network television (pretend it was passed a couple of years ago) on the number of sexual crimes committed in the United States. What type of study might be best? A. small n design B. field experiment C. interrupted time series design D. nonequivalent control group design E. longitudinal study

Next week Comment on readings AND Find and evaluate a meta-analysis on a topic that interests you Rough drafts due post-spring break