1. Sampling and Sample Size Part 1 Cally Ardington

2. Course Overview 1.What is Evaluation? 2.Outcomes, Impact, and Indicators 3.Why Randomise? 4.How to Randomise? 5.Sampling and Sample Size 6.Threats and Analysis 7.Project from Start to Finish 8.Cost Effectiveness and Scaling

3. Lecture Outline Precision and accuracy Statistical tools  Population and sampling distribution  Law of Large Number and Central Limit Theorem  Standard deviation and standard error

4. Which of these is more accurate? A. I. B. II. C. Don’t know I. II.

5. Accuracy versus Precision Precision (Sample Size) Accuracy (Randomization) truth estimates

6. Accuracy versus Precision Precision (Sample Size) Accuracy (Randomization) truth estimates truth estimates

7. This session’s question How large does the sample need to be for you to be able to detect a given treatment effect? Randomization removes the bias (ensures accuracy) but it does not remove noise We control precision with sample size

8. Lecture Outline Precision and accuracy Statistical tools  Population and sampling distribution  Law of Large Number and Central Limit Theorem  Standard deviation and standard error

9. Population distribution

10. Take a random sample : Sampling distribution

11. Lecture Outline Precision and accuracy Statistical tools  Population and sampling distribution  Law of Large Number and Central Limit Theorem  Standard deviation and standard error

12. We generally don’t have a our population distribution but, we have our sampling distribution. What do we know about our sampling distribution? Two statistical laws help us here (1)Central Limit Theorem (2)The Law of Large Numbers

13. (1) Central Limit Theorem To here… This is the distribution of the population (Population Distribution) This is the distribution of Means from all Random Samples (Sampling distribution)

14. Central Limit Theorem Population Draw 1 Mean test score Draw 2 Mean test score Draw 3 Mean test score

15. Central Limit Theorem Population Draw 6 Mean test score Draw 5 Mean test score Draw 4 Mean test score

16. Central Limit Theorem Population Draw 9 Mean test score Draw 10 Mean test score Draw 8 Mean test score Draw 7 Mean test score

17. Draw 10 random students, take the average, plot it: 10 times. Inadequate sample size No clear distribution around population mean

18. More sample means around population mean Still spread a good deal Draw 10 random students: 50 and 100 times

19. Distribution now significantly more normal Starting to see peaks Draws 10 random students: 500 and 1000 times

20. This is a theoretical exercise. In reality we do not have multiple draws, we only have one draw. BUT, we can control the number of people in that draw. This is what we refer to as SAMPLE SIZE. The previous example was based on a sample size of 10 What happens if we take a sample size of 50?

21. What happens to the sampling distribution if we draw a sample size of 50 instead of 10, and take the mean (thousands of times)? A.We will approach a bell curve faster (than with a sample size of 10) B.The bell curve will be narrower C.Both A & B D.Neither. The underlying sampling distribution does not change.

22. N = 10 N = 50 (2) Law of Large Numbers

23. N= 10 N = 50

24. Lecture Outline Precision and accuracy Statistical tools  Population and sampling distribution  Law of Large Number and Central Limit Theorem  Standard deviation and standard error

25. Standard deviation/error What’s the difference between the standard deviation and the standard error? The standard error = the standard deviation of the sampling distributions

26. Variance and Standard Deviation

27. Standard Deviation

28. Standard Error

29. Sample size ↑ x4, SE ↓ ½ Sample Frequency Population mean Standard deviation Standard error Sample Distribution

30. Sample size ↑ x9, SE ↓ ?

31. Sample size ↑ x100, SE ↓? Sample Frequency Population mean Standard deviation Standard error Sample Distribution