1. Scientific Utopia: I. Improving Scientific Communication Brian Nosek University of Virginia Center for Open Science

2. PURPOSE OF SCIENCE To acquire knowledge about nature

3. COMMUNICATION PROBLEMS None Slow Incomplete Inaccurate Unmodifiable

4. Objections Quality control Incentive structure Information overload Ego protection Status quo bias Too many with a stake in current system Commercial interests for secrecy Financial benefits for societies

5. NO MORE PAPER Solution: DEMO Problem: Scientific communication is inefficient

6. OPEN ACCESS Solution: DEMO Problem: Scientific communication is inefficient CoN

7. Public University Granting Agencies Scientists Publishers pays for doing research Gives ownership of research report, free Pays to access the research

8. SEPARATE PUBLICATION FROM EVALUATION Solution: Problem: Scientific communication is inefficient DEMODEMO 1 DEMODEMO 2

9. GRADING SYSTEM + DIVERSIFIED DISSEMINATION (END THE 1 JOURNAL MODEL) Solution: Problem: Scientific communication is inefficient DEMODEMO 1 DEMODEMO 2

14. OPEN PEER REVIEW Solution: DEMO Problem: Scientific communication is inefficient

15. CONTINUOUS PEER REVIEW Solution: DEMO Problem: Scientific communication is inefficient

16. Closing – the new reality

17. Acknowledgments Yoav Bar-Anan Members of the implicit cognition lab

18. Part 2 preview 1.No more am(bem)iguity between confirmatory and exploratory research 2.No more unpublished research 3.No more materials or data left out 4.No more articles http://openscienceframework.org/

19. UniversitiesSocieties Individuals Commercial Publishers

20. Source of Inefficiency Mixing exploratory and confirmatory research + lack of emphasis on replication + emphasis on rejecting the null + lack of reporting null effects = literature with an overrepresentation of false positives + little chance of correcting false positives once published

21. Addressing the inefficiency with a Study Registry 1.Clearly identify the degree to which research is exploratory versus confirmatory – reduce false-positive, increase value of replication 2.Clearly identify what research has been done, whether or not it results in a published article – eliminate file-drawer problem

22. NO MORE ARTICLES Solution: DEMO Problem: Scientific communication is inefficient

23. The Empirical Research Process * with the “study” as the unit of analysis * CONCEPTUALIZATIONEXECUTIONEVALUATION 1.Theory/Background Research 2.Question/Hypothesis 3.Methods/Design 4.Analysis Plan a.Variable and response coding plan b.Cleaning/exclusion/ transformation plan c.Tests and evaluation plan 1.Sample Identification 2.Sample Recruiting 3.Data Collection 1.Analysis 2.Interpretation 3.Report Time

24. NO MORE MATERIALS OR DATA LEFT OUT Solution: DEMO Problem: Scientific communication is inefficient

25. Assumptions 1.Academic science is more efficient as an open, shared process 2.The current system is much more closed than it needs to be 3.The current incentive structure for individual contributors does not encourage openness 4.Current infrastructure does not make it easy to be open or share 5.Infrastructure can assist openness and increase sharing, leading to more efficient (and rapid) knowledge accumulation 6.Infrastructure can also make data accumulation, aggregation, and analysis more efficient

26. Macro Goals 1.Develop infrastructure to make openness and sharing effortless 2.Develop infrastructure, recognition systems, and process models to create incentives for individual contributors for openness and sharing 3.Develop infrastructure to make data accumulation, aggregation, and analysis more efficient

28. VALUES OF SCIENCE Transparency Accuracy Sharing Reproducibility Cumulative Progress Economy / Efficiency

29. Peer Review Submit for publication Draft article Analyze data Conduct a study Article published New research inspired Design a study