Next week 2 book chapters Outline of thesis proposal/paper intro Find a scale and answer questions Thought paper
General Sampling Issues What is the sampling model? What types of biases can come in at each point? What is the proximity similarity model? What are issues with that model? How can you increase external validity? When do you need a representative sample?
Sampling Distributions How are sampling distributions relevant to research? What is the difference between the variance, standard deviation, and standard error? How does the standard error relate to n? SD? What do 68, 95, and 99 refer to? What are confidence intervals? What do they mean?
Probability Sampling Are random samples really random? When would you use each? Simple random sampling Stratified random Systematic random Cluster Multistage
Nonprobability Sampling What are these? When should they be used? Convenience sample Modal instance Expert Quota Heterogeneity Snowball
Census How do censuses do sampling? What are problems with the national census? Ways to deal with them?
Power What is power, and why does it matter? Why do studies get published even if they are underpowered? What group-level consequence (for science) does this have? How do people determine power? How should they?
Power vs. accuracy How does sample size planning for power vs. accuracy differ? When would you want to do one vs. the other? Figure 1. For AIPE, effect size doesn’t matter
Power What stats should we report in a study? What does APA manual say? How can simulations be used to estimate power? How does power relate to meta-analyses? How does power differ for omnibus vs. specific tests? What’s post hoc power? What’s the problem with it? For your papers in here, estimate power and justify your sample size (don’t just plug into g-power)
Other stat issues The problem of p http://www.youtube.com/watch?v=ez4DgdurRPg http://www.youtube.com/watch?v=ez4DgdurRPg The relationship between p, effect size, and n Practical vs. statistical significance How much power should we have? Registries, multi-site trials Standardized vs. unstandardized effect sizes
How can you increase power? Increase sample size or alpha Decrease mean square error by using better measures, increasing control, and getting high quality data Use within-participant designs or use covariates Increase the variance of the IV (use a more powerful treatment) Use orthogonal contrasts or get predictors that aren’t correlated to each other Ensure that you’re not violating assumptions of your stats Look at theory and previous research to find the best, most powerful predictors Use a more homogeneous sample Do field studies Increase sample size Treat missing data in a more appropriate way McClelland, 2000; Funder et al., 2014
Funder et al., 2014 SPSP Task Force on Publication and Research Practices Recommendations: Describe choice of N and issues of power Report effect sizes and 95% CI Avoid “questionable research practices” Give all IV and DV instructions and measures in an Appendix or online Provide data and coding to those who ask
Get better outlets for replication studies Be open to differences in methods, groups, etc. Recommendations for education: Encourage “getting it right” over “finding significant results” (p. 9) Tell the “whole story” rather than a “good story” (p. 9) Teach things like power, effect sizes, CI, questionable practices, etc. Be good models
Cross-cultural research What is a culture? Why study things across cultures? Best practices: Have at least one insider on the research team Match samples on typicality and as many things as you can besides culture Translation and back translation Are they cultural differences or miscommunications or differences of response styles?
Henrich, Heine, & Norenzayan, 2010 Sears, 1988 What are WEIRD samples? How much can we generalize our results? When does generalization make sense? Why do we focus so much on WEIRD samples? What should we report about demographics?
Industrialized societies vs. small-scale societies What kinds of differences exist? Similarities? Why?
Western vs. non-Western What are differences? Similarities? Why?
Americans vs. other Westerners How are we weird? Why are we weird? What does this suggest about what we “know” in psychology?
American participants vs. others Differences? How do college students differ from others? For what topics are less likely or not to have an effect? How do we differ from Americans in the past?
Overall What problems do WEIRD samples cause? When is it okay to use WEIRD samples? How can we deal with these issues? So is your research just a worthless pile of ….?