Data Analysis Econ 176, Fall 2014

Populations When we run an experiment, we are always measuring an outcome, x. We say that an outcome belongs to some population of outcomes. Population of the behavior of undergrad students participating in a particular economic system with a particular set of parameters. If we changed something about the experiment, we would be looking at a different population. Usually assume our population is representative in some way. Of any human being who could be in the experiment. Of outcomes in the economic systems we model with our experiment (external validity).

Populations Implicit in the idea of a population of observations, is the expectations some variation In the population of “students playing the battle of the sexes game” I would expect some people to act differently than others. We usually want to know what the average outcome is for a population. Average behavior. On average how much do people contribute to public goods? On average how often do pairs play the payoff dominant equilibrium? On average how often does behavior converge to a competitive equilibrium? On average how rational are people’s use of imperfect information?

Sampling Error We are usually blocked from directly observing the actual average of the population (called the population mean) We usually only get a small sample of the population. We don’t get to see everyone’s behavior -- just the few people who participate in our experiment. As a result, the average behavior we observe could be (by pure chance) different from the actual average behavior overall for the population. We call this (very real) possibility, sampling error. The reason we look at many observations.

Sampling Error As we get larger and larger samples from a population, we can be more and more sure our average behavior is like the population’s average behavior. In the limit, sampling error goes away, and we have perfectly reliable results Statistics says that we can be more confident in our results, the more independent observations we have in our sample.

Independent Trials What counts as an independent trial? Technically, an observation (or set of observations) in our sample that doesn’t depend on: Other observations from our sample. Causes that are also causing other observations. Why not take 1000 observations on one person? A single subject? A single group of subjects playing at once?

Informal and Formal Data Analysis There are two stages of data analysis: Informal and Formal. Informal: Look at summary statistics. Look at summary graphs. Look for obvious patterns, tendencies, comparisons. Formal: Do statistical analysis. “Prove” obvious patterns you see. Look for non-obvious patterns.

Tools Microsoft Excel (part of Office) has a lot of basic tools. Analysis Took Pack Make graphs. Do statistical tests. More advanced statistical packages. Stata SAS EViews

Measurement Often economic theory predicts behavior should look like something specific. A first price sealed bid auction bid should be: People should contribute 0 in public goods situations. Most of the time, the “payoff dominant equilibrium will be chosen.” People shouldn’t exhibit trust in non-repeated interactions. We often want to look at the data and discover whether behavior in the population matches well with these predictions. Often we want to measure things that theory tells us nothing much about.

A Bargaining Market Imagine we have six buyers and six sellers with values and costs and no knowledge of the values and costs of others. The institution for trade: people walk around and negotiate. Each period they trade until nobody wants to trade anymore. An experimental question: How efficient was trade? We run 3 sessions. Each session consists of 10 periods of trade.

Looking at Averages We are interested in what tends to happen in the population. Look at the average efficiency over all rounds for each session. It looks like something strange was happening in session 1 -- a much larger efficiency?

Looking at Averages No…the graph simply wasn’t scaled very well. It looks like average behavior is relatively consistent across sessions. So, based on this sample, what do we estimate to be in the population?

Looking at Averages Based on the data so far, it looks like average efficiency is just over 60%. We add an error bar -- the standard deviation divided by the square root of observations. Give us a sense of how much behavior varies across our observations.

Looking at Distributions Generally we want to get a feel for exactly how much behavior varies. Tells us how precise our estimates are. Can use a histogram. Shows frequency of observations in different ranges. How dispersed is the data?

Looking at Trends Often we want to look at how behavior changes over time. Take same data from histogram and divide it up by session and period. Look for: Experience effects Learning

Looking at Trends We can look at the average over sessions for each period. Gives us an estimate in each period rather than overall. So it looks like there is some sort of experience effect at work here.

Comparison Whenever we design a treatment (vary a factor) we do so to compare measurements on two or more populations. Every treatment has its own population of possible behavior. What sort of comparisons do we make? Efficiency in double auctions vs. posted offer markets. Coordination when battle of the sexes outcomes are close to equal vs. really unequal. Effects of chat-room on behavior in game theory games. Bidding behavior in first price versus English auctions.

Comparing Institutions What happens to efficiency when we change the institution to a double auction? Imagine, again, we have six buyers and six sellers with values and costs and no knowledge of the values and costs of others. Each period they trade in a double auction An experimental question: Does the change in the institution alter efficiency? We run 3 more sessions, under a new institution, with new subjects. Each session, again, consists of 10 periods of trade. Now we are comparing behavior in treatments.

Comparing Averages First we can look at the average efficiency in each session, for both treatments. Compare the natural variation within a treatment seems to be similar to variation between treatments. Does the treatment have too big of an effect to be chalked up to natural variation?

Comparing Averages We can also look at the overall average in each treatment. Do these averages look different? The error bars (showing standard deviation) give us some sense of the effect of the treatment relative to normal variation in observed behavior.

Comparing Distributions Another way to compare the overall variation to the variation due to treatment is to look at side by side histograms. If they “overlap” too much, we might suspect the behavior in the two treatments come from the same distribution. That is, the treatment doesn’t have an effect. We don’t have that “problem” here!

Comparing Trends We can also compare the trends across periods. Do behaviors converge? Does learning happen in both treatments? Does one institution require more experience to use well?

Comparing Trends We can compare the averages over sessions for each period and treatment. Don’t converge. Experience seems to matter for the Bargaining treatment but not for the Market treatment.