Presentation is loading. Please wait.

Presentation is loading. Please wait.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. Lecture 23: Experiments (Chapter 15.1–15.5)

Similar presentations


Presentation on theme: "Copyright © 2006 Pearson Addison-Wesley. All rights reserved. Lecture 23: Experiments (Chapter 15.1–15.5)"— Presentation transcript:

1 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. Lecture 23: Experiments (Chapter 15.1–15.5)

2 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-2 Agenda Review Negative Income Tax Experiments Estimating Means (Chapter 15.1) Randomized Experiments (Chapter 15.2, 15.5) Natural Experiments (Chapter 15.3) Differences-in-Differences (Chapter 15.4)

3 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-3 Review Under the Gauss–Markov assumptions, OLS is consistent, unbiased, and efficient. Under heteroskedasticity or serial correlation, OLS is still consistent and unbiased, but inefficient (and the OLS formula for estimated standard errors is incorrect) When X is correlated with, then OLS is inconsistent and biased.

4 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-4 Review (cont.) There are many possible reasons why X could be correlated with : – Omitted Variables – Simultaneity – Measurement Error

5 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-5 Review (cont.) Solutions to contemporaneous correlation must break the link between X and. Instrumental Variables breaks this link ex post, isolating part of the variation observed in X that is known to be uncorrelated with. Experiments break the link ex ante.

6 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-6 Example: Negative Income Tax In the 1970s, the government conducted extensive research on a guaranteed minimum income. Individuals with incomes lower than the minimum would have their incomes supplemented by the tax system (a negative income tax).

7 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-7 Example: Negative Income Tax (cont.) What would be the effects of an NIT? Would the guaranteed income lead to mass emigration from the labor force to public assistance, as low-wage workers lost the incentive to work? Policy makers wanted a clear answer, free of biases, and were willing to pay for a massive research program.

8 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-8 Example: Negative Income Tax (cont.) In four separate NIT experiments, over eight thousand households were randomly assigned to receive guaranteed incomes. Because the assistance rate was assigned randomly, researchers knew it was uncorrelated with.

9 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-9 Example: Negative Income Tax (cont.) The NIT experiment showed that a guaranteed minimum income had moderate effects on labor effort. Heads of households who faced the highest effective tax rate (because if they earned more money they would reduce their subsidy) decreased hours worked by about 11%-15% Second earners cut their hours of work more dramatically, by 20-30%

10 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-10 Example: Negative Income Tax (cont.) Randomized social policy experiments have also been used to study: – Class size (the Tennessee STAR experiment) – Housing subsidies (HUDs Moving to Opportunity study) – Job training programs (the Department of Labors Job Training and Partnership Act assessments)

11 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-11 Example: Negative Income Tax (cont.) Randomized social policies are very expensive, and so are conducted infrequently. Laboratory studies are becoming increasingly common (using undergraduate subjects in simulated settings, almost always playing for real but small amounts of money). Laboratory studies test economic theory but do not estimate the effects of social programs.

12 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-12 Example: Negative Income Tax (cont.) The basic methodological principle of the NIT research is quite simple: if we assign X directly, then we can eliminate its relationship to (at least at the point of assignment).

13 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-13 Estimating Means (Chapter 15.1) In principle, experimental economists can assign a wide range of X values and use OLS methods to estimate coefficients. Small-scale laboratory experiments can easily assign a range of X values. However, for larger studies, it is often practical to work with only a few X values. The NIT studies are an exception.

14 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-14 Estimating Means (cont.) When X takes on only a few values, the parameter of interest shifts from estimating a slope to estimating a mean. We have already seen that dummy variables can be used to estimate different means for different groups. Now we will look more closely at estimating means.

15 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-15 Estimating Means (cont.) Typically we are interested in knowing the effect of one or more treatments. We want to know what outcomes are caused by the treatment/s. For example, what effect does the Head Start program have on participants grades? What effect would moving to a better neighborhood have on residents of impoverished neighborhoods?

16 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-16 Estimating Means (cont.) We really want to know the causal effect of the treatment T. We want to know what would happen on average to a person randomly chosen from the population if we gave him/her the treatment, as opposed to NOT giving him/her the treatment.

17 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-17 Estimating Means (cont.) Unfortunately, we do not get to observe the same individuals in each state. A naïve analyst might simply compare the outcomes for those observed in the treatment state to those not observed in the treatment state. In an observational study, we cannot DIRECTLY compare groups that receive the treatment to groups that do not.

18 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-18 Estimating Means (cont.) Unfortunately, we do not get to observe the same individuals in each state. In an observational study, we cannot DIRECTLY compare groups that receive the treatment to groups that do not. Families that choose to join programs like Head Start differ systematically from families that do not join such programs.

19 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-19 Estimating Means (cont.) Whenever selection into a treatment is non-random, researchers must worry about unobserved heterogeneity among subjects. Some subjects have greater ability, motivation, resources, etc., that make them more likely to seek out and gain access to helpful treatments (and to avoid unhelpful ones).

20 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-20 Estimating Means (cont.) Treatments also tend to attract individuals who derive the most benefit from them. Public housing is highly unattractive. People in public housing tend to be those with the worst access to alternatives.

21 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-21 Estimating Means (cont.) Selection Bias: individuals are sorted into the treatment/non-treatment group on the basis of some underlying characteristic, such as motivation. This underlying characteristic has its own effect on outcomes. More motivated parents enroll their children in Head Start. Children of motivated parents do better academically.

22 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-22 Estimating Means (cont.) A naïve comparison would not estimate but would come closer to estimating

23 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-23 Estimating Means (cont.) The selection bias is a special case of omitted variables bias. The goal of experiments is to break the selection bias.

24 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-24 Estimating Means (cont.) Breaking the selection bias requires the experimenter to intervene in the agents world in some way. The greater the intervention, the greater the control the economist possesses, and the more certain the economist is to eliminate selection biases. The greater the intervention, the greater the danger that the results will not generalize to a more authentic situation.

25 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-25 Randomized Experiments (Chapter 15.2, 15.5) The experimenter randomly divides the subjects into two groups, a treatment group and a control group. The treatment group receives the treatment ( T = 1). The control group does not receive the treatment ( T = 0).

26 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-26 Randomized Experiments (cont.) Individual subjects will vary in myriad unobservable ways. By randomly assigning the treatment, the experiment ensures that on average these differences will cancel out. More motivated subjects are as likely to be in the control group as in the treatment group.

27 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-27 Randomized Experiments (cont.) Because the treatment has been randomly assigned, the treatment and control groups are on average the same (within the bounds of sampling error). The only systematic difference between the two groups is the treatment. The effect of the treatment can be estimated by comparing the two groups means.

28 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-28 Randomized Experiments (cont.) Example: the Moving to Opportunity Program – HUD randomly assigns a sample of low-income participants to receive a special housing voucher (the treatment group) or to receive no voucher (the control group)

29 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-29 Randomized Experiments (cont.) The vouchers are only valid for housing in good neighborhoods. Receipt of the voucher is not correlated with any other traits of the participants. Actual use of the voucher will be correlated with such traits as motivation, income, and existing connections in a qualifying good neighborhood.

30 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-30 Randomized Experiments (cont.) Researchers must be very careful about which effects have been successfully isolated from, and which have not. The Moving to Opportunity experiment DOES provide an unbiased estimate of the effect of giving low income families a voucher to move to a good neighborhood. Such estimates are valuable for assessing proposals for such a voucher program.

31 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-31 Randomized Experiments (cont.) The experiment does NOT provide an unbiased estimate of the effects of living in a good neighborhood vs. a poor neighborhood. Not every subject assigned to the treatment group will take advantage of the treatment. Treatment subjects will self-select into those who use the treatment, and those who do not.

32 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-32 Randomized Experiments (cont.) Suppose we define the treatment as moving from a poor neighborhood to a good neighborhood. If the participants in the program are representative of the population of interest, and if everyone who receives a voucher uses it, then we can estimate the average effect of the treatment.

33 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-33 Randomized Experiments (cont.) Suppose we define the treatment as moving from a poor neighborhood to a good neighborhood. If the participants in the program are NOT representative of the population of interest, and if everyone who receives a voucher uses it, then we can estimate the average effect of the treatment on the treated.

34 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-34 Randomized Experiments (cont.) Suppose we define the treatment as moving from a poor neighborhood to a good neighborhood. If NOT everyone who receives a voucher uses it, then we can estimate the average effect of the treatment on those we intend to treat.

35 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-35 Randomized Experiment (cont.) Biases in Randomized Experiments: – Non-Response Bias: participants do not provide their data to the researchers – Attrition Bias: participants drop out of the study – Sample Selection Bias: individuals who agree to participate in a randomized study differ from the population of interest

36 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-36 Randomized Experiments (cont.) General Equilibrium Effects: the experiment shows the results of a small-scale program. Implementing the program on a larger scale might change the environment in ways that a smaller scale study does not.

37 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-37 Randomized Experiments (cont.) Impoverished neighborhoods are not greatly affected by the emigration of a few of their most talented residents from a small study. If HUD decides to implement the program on a wide scale, poor neighborhoods could be greatly affected.

38 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-38 Randomized Experiments (cont.) Another example of General Equilibrium Effects: training programs If a few individuals randomly receive extra job training, their wages increase because (i) they are more productive and (ii) they have a competitive edge. If everyone receives the extra training, no one gains a competitive edge.

39 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-39 Natural Experiments (Chapter 15.3) Researchers must choose how much control to exert over their experiment. Greater control decreases any possible correlations between the treatment and. For example, HUD could have (at great expense) moved everyone in the treatment group to a good neighborhood, instead of relying on participants to choose how to use vouchers.

40 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-40 Natural Experiments (cont.) In a laboratory experiment, experimental economists can exert great control over every aspect of the subjects environment. Greater control increases the artificiality of the experiment. At some point, economists wish to generalize from the subjects and environments studied to a real program in the population of interest.

41 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-41 Natural Experiments (cont.) Internal Validity: the ability of the economist to attribute differences between the treatment and control groups to the treatment itself ( X and are uncorrelated). External Validity: the ability of the economist to generalize from the experiment to the setting and population of interest.

42 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-42 Natural Experiments (cont.) Often economists face a trade-off between internal and external validity. The more they break the natural connections between X and, the more danger arises that the results will not generalize. Natural experiments often offer greater external validity (and are much cheaper!)

43 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-43 Natural Experiments (cont.) A natural experiment is an observational study of a natural setting that appears to assign a treatment in a reasonably random manner. Natural experiments have great external validity, at least to the particular setting and population affected. Natural experiments have weaker internal validity; the experiment does not control the randomization process directly.

44 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-44 Natural Experiments (cont.) Example: medical residency clearinghouses The American Medical Association runs a centralized clearinghouse to assign medical school graduates to hospitals for residency training programs. Hospitals and applicants each submit ranked preference lists, and the AMA assigns matches.

45 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-45 Natural Experiments (cont.) The AMA hired economist Alvin Roth to redesign their matching procedures. Roth knew from game theory that a matching algorithm would work better if it were stable, i.e. if no residency/hospital pair would be better off by abandoning their assigned matches to match with each other. But is this property really important?

46 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-46 Natural Experiments (cont.) Roth observed a natural experiment in the United Kingdom. In the UK, each region conducts its own regional clearinghouse. Some regions adopted matching procedures that were stable (i.e. that had desirable game theoretic properties); others did not.

47 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-47 Natural Experiments (cont.) The Treatment: using a stable matching procedure The Treatment Group: regions in the UK that had adopted stable procedures The Control Group: regions in the UK that had NOT adopted stable procedures

48 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-48 Natural Experiments (cont.) The Outcome: regions in the Treatment group were still using their matching procedures several years later. Regions in the Control group had abandoned their clearinghouses.

49 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-49 Natural Experiments (cont.) However, Roth could only HOPE that the two groups differed only in their choice of procedures. Perhaps the Treatment group regions were simply better at running clearinghouses, or had cultural differences? The experiments internal validity was suspect. Roths solution: a laboratory experiment.

50 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-50 Natural Experiments (cont.) Roth designed a simple matching market that could be run in a few minutes by undergraduate subjects on computers. Some undergraduates pretended to be hospitals; others pretended to be applicants. Undergraduates were paid based on their ability to make a good match.

51 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-51 Natural Experiments (cont.) In some randomly chosen laboratory markets, the clearinghouse used the stable rules adopted by the Treatment group in the natural experiment. Other laboratory markets used the bad rules adopted by the Control group in the natural experiment. The laboratory subjects reached the same outcome as the real UK regions.

52 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-52 Natural Experiments (cont.) In Roths randomized laboratory experiment, the Treatment and Control groups differed ONLY in the rules used. However, the laboratory study used a very abstract computer game and undergraduate subjects. External validity is suspect.

53 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-53 Natural Experiments (cont.) The laboratory experiments, with their strong internal validity, suggest that good game theoretic rules are a major factor in the success of clearinghouses. The natural experiments, with their strong external validity, suggest that the rules are important in real markets similar to the one the AMA wished to design. The two studies reinforce each other.

54 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-54 Natural Experiments (cont.) Another example: the Minimum Wage – Policy makers have long argued over the effects of minimum wage laws – Benefits: some workers receive higher wages – Costs: businesses must pay higher wages; some workers become unemployed when employers decide not to pay the higher wage

55 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-55 Hit Figure 15.1 The Nominal and Real Minimum Wage

56 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-56 Natural Experiments Card and Krueger uncovered a natural experiment to study the effect of a minimum wage increase on the employment of low-skill teenagers in the fast food industry. In 1992, New Jersey rose its state- mandated minimum wage from $4.25 to $5.05. Fast food workers in NJ naturally became a treatment group.

57 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-57 Natural Experiments (cont.) Card and Krueger needed a control group. They chose to focus on fast food workers in neighboring Pennsylvania, which did not increase its minimum wage law. NJ and PA are very similar states. Teenagers families decisions to live in one or the other are very unlikely to be correlated with NJs decision to raise its minimum wage in 1992.

58 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-58 Natural Experiments (cont.) After the change in the minimum wage, fast food restaurants in NJ employed on average 30.0 workers. In our control state of PA, restaurants employed 30.9 workers. At first glance, it appears that the higher minimum wage state has slightly smaller employment in fast food restaurants.

59 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-59 Natural Experiments (cont.) One problem: NJ and PA are similar, but not quite identical. Before NJs law changed, fast food restaurants in NJ employed on average 29.8 workers. PA restaurants employed 33.1 workers. We need to take these baseline differences into account.

60 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-60 Natural Experiments (cont.) PA is not quite identical to NJ Perhaps we should look for a different control group. We could use NJ restaurants before the law changed. These are the exact same restaurants as in the treatment group, so they should be identical.

61 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-61 Natural Experiments (cont.) Problem: there are other differences between early 1992 and late 1992 than just the change in the law. Perhaps the macroeconomy improved. Perhaps restaurants increased employment because of seasonal variation in the consumption of fast food.

62 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-62 Natural Experiments (cont.) The Solution: look at the change in employment in both NJ and PA when the NJ law was enacted. Employment per restaurant in NJ increased from 29.8 to 30.0 Employment per restaurant in PA fell from 33.1 to 30.9

63 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-63 HIT TABLE 15.1 Diff-in-Diffs Estimate of the Employment Effect of New Jerseys Minimum Wage Increase

64 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-64 Natural Experiments NJ and PA do have some differences, but we are HOPING those differences do not change from early 1992 to late 1992. Conditions for fast food restaurants do change from early 1992 to late 1992, but we are HOPING those differences are the same in both NJ and PA (except for the NJ minimum wage increase).

65 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-65 Natural Experiments (cont.) In effect, to interpret our natural experiment, we have used TWO control groups. Neither control group is perfect, but using the two together HOPEFULLY eliminates the imperfections. This procedure of using two control groups is called Differences-in-Differences (or Diffs-in-Diffs).

66 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-66 Differences-in-Differences (Chapter 15.4) Let us construct a more general DGP. The outcome, Y i, is the result of three factors: 1.Receiving the treatment (e.g. a minimum wage increase) 2.Being in the group that receives the treatment (e.g. NJ) 3.Being in the time after the treatment has occurred (e.g. late 1992)

67 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-67 Differences-in-Differences (cont.) where D Treatment = 1 if the observation is of a subject assigned to the treatment group, either before or after the treatment is received and D After = 1 if the observation is of a subject in either group after the treatment has been received by the treatment group

68 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-68 Differences-in-Differences (cont.) 1 captures the underlying difference between the treatment and control groups. 2 captures the underlying difference between the two time periods. 3 captures the effect of the treatment.

69 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-69 Differences-in-Differences (cont.)

70 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-70 Differences-in-Differences (cont.) One method for estimating 3 is to calculate the means for each group at each time and subtract twice.

71 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-71 Differences-in-Differences (cont.) However, we want to calculate an estimated standard error for the treatment group. This calculation is not transparent when subtracting group means. We also might want to add other explanators. Can we express Diffs-in-Diffs in a regression framework?

72 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-72 Differences-in-Differences (cont.) We can estimate group means using dummy variables.

73 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-73 Checking Understanding

74 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-74 Checking Understanding (cont.)

75 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-75 Differences-in-Differences (cont.) The Great Weakness of Diffs-in-Diffs – Some other difference may arise between the Treatment and Control groups at the same time that the Treatment occurs. – Example: suppose at the same time NJ increased the minimum wage, PA introduced a new food labeling law that reduced the demand for fast food.

76 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-76 Review We are often interested in knowing the effect of one or more treatments. We want to know what outcomes are caused by the treatment/s.

77 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-77 Review (cont.) We really want to know the causal effect of the treatment T. We want to know what would happen on average to a person randomly chosen from the population if we gave him/her the treatment, as opposed to NOT giving him/her the treatment.

78 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-78 Review (cont.) Whenever selection into a treatment is non-random, researchers must worry about unobserved heterogeneity among subjects. Some subjects have greater ability, motivation, resources, etc., that make them more likely to seek out and gain access to helpful treatments (and to avoid unhelpful ones).

79 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-79 Review (cont.) Treatments also tend to attract individuals who derive the most benefit from them.

80 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-80 Review (cont.) Selection Bias: individuals are sorted into the treatment/non-treatment group on the basis of some underlying characteristic, such as motivation. This underlying characteristic has its own effect on outcomes.

81 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-81 Review (cont.) The experimenter randomly divides the subjects into two groups, a treatment group and a control group. The treatment group receives the treatment ( T = 1). The control group does not receive the treatment ( T = 0).

82 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-82 Review (cont.) Individual subjects will vary in myriad unobservable ways. By randomly assigning the treatment, the experiment ensures that on average these differences will cancel out. More motivated subjects are as likely to be in the control group as in the treatment group.

83 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-83 Review (cont.) Because the treatment has been randomly assigned, the treatment and control groups are on average the same (within the bounds of sampling error). The only systematic difference between the two groups is the treatment. The effect of the treatment can be estimated by comparing the two groups means.

84 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-84 Review (cont.) If our subject pool is the same as the population of interest, and the entire Treatment group responds to the Treatment, we can estimate the average effect of the Treatment. If our subject pool differs from the population of interest, but the entire Treatment group is responsive, we can estimate the average effect of the Treatment on the Treated.

85 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-85 Review (cont.) If not all of our Treatment Group responds to the Treatment, then we estimate the average effect of the treatment on those we intend to treat.

86 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-86 Review (cont.) Biases in Randomized Experiments: – Non-Response Bias: participants do not provide their data to the researchers – Attrition Bias: participants drop out of the study – Sample Selection Bias: individuals who agree to participate in a randomized study differ from the population of interest

87 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-87 Review (cont.) General Equilibrium Effects: the experiment shows the results of a small-scale program. Implementing the program on a larger scale might change the environment in ways that a smaller scale study does not.

88 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-88 Review (cont.) Internal Validity: the ability of the economist to attribute differences between the treatment and control groups to the treatment itself ( X and are uncorrelated). External Validity: the ability of the economist to generalize from the experiment to the setting and population of interest.

89 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-89 Review (cont.) Often economists face a trade-off between internal and external validity. The more they break the natural connections between X and, the more danger arises that the results will not generalize. Natural experiments often offer greater external validity.

90 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-90 Review (cont.) A natural experiment is an observational study of a natural setting that appears to assign a treatment in a reasonably random manner. Natural experiments have great external validity, at least to the particular setting and population affected. Natural experiments have weaker internal validity; the experiment does not control the randomization process directly.

91 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-91 Review (cont.) Suppose the outcome, Y i, is the result of three factors: 1.Receiving the treatment (e.g. a minimum wage increase) 2.Being in the group that receives the treatment (e.g. NJ) 3.Being in the time after the treatment has occurred (e.g. late 1992)

92 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-92 Review (cont.) where D Treatment = 1 if the observation is of a subject assigned to the treatment group, either before or after the treatment is received and D After = 1 if the observation is of a subject in either group after the treatment has been received by the treatment group

93 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-93 Review (cont.) 1 captures the underlying difference between the treatment and control groups. 2 captures the underlying difference between the two time periods. 3 captures the effect of the treatment.

94 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-94 Review: Diffs-in-Diffs

95 Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 23-95 Review The Great Weakness of Diffs-in-Diffs – Some other difference may arise between the Treatment and Control groups at the same time that the Treatment occurs. – Example: suppose at the same time NJ increased the minimum wage, PA introduced a new food labeling law that reduced the demand for fast food.


Download ppt "Copyright © 2006 Pearson Addison-Wesley. All rights reserved. Lecture 23: Experiments (Chapter 15.1–15.5)"

Similar presentations


Ads by Google