1. Data-Intensive Research: Making better use of Research Data Malcolm Atkinson & David De Roure mpa@nesc.ac.uk & dder@ecs.soton.ac.uk 8 December 2009 Report from a fact-finding mission

2. Mission goal: learn how researchers use data 2 Acknowledgements: the UK e-Science Directors CIR authors, our teams, the EPSRC & all our hosts in the USA, they had the good ideas; all the opinions, observations and recommendations are our own.

3. Outline Cornucopia of data Yet to learn how to use it well Hot topic Research to politics Concepts Datascopes, Intellectual Ramps, Going the last mile Co-* Digital ecosystem Principles Recommendations Actions Survival in the Digital Revolution 3

4. Data-Intensive Research Events Bermuda agreement 1996, 97 & 98 SDSS Archive DB1999 Human Genome 2001 DI Comp. Environm’s2001 Fort Lauderdale2003 Hey&Trefethen D.Del.2003 Digital Curation Cen.2004 NSF DataNet call2007 XLDB series starts2007 SciDB starts2008 Yahoo DI workshop2008 Harnessing data2009 Beyond data del.2009 Gov’s use Linked D.2009 NSF CISE DI call2009 4 th Paradigm book2009 JISC Research DM2009 e-IRG DMTF report2009 DIEW Japan2010

5. Sir Tim Berners-Lee http://www.w3.org/DesignIssues/GovData.html

6. Datascopes for the mind 6 NRAO/AUI/NSF To see things in your data you could never see before Data to Information to Knowledge to Wisdom Changed our place in the universe

7. Example datascopes 7

8. Searching for an expression ≈ OTX1 AND (Pou3f2 OR (Brd4 AND Sim1)) 2903 ≈ 947 AND (1688 OR (1697 AND 3096)) Dmbx1 Match = 0.755 Slide from Jano van Hemert

9. Intellectual ramps 9 Easy and low risk to start Progress to advanced skills For research data users No obligation Go as far as you want Find a service & relax

10. Dropbox as a Ramp Local folder synchronised and shared via cloud Condor job submitted by drag and drop Ian Cottam Results appear in Dropbox Slide from David De Roure

11. Intuitive interfaces e-Science Research http://research.nesc.ac.uk/rapid/ Slide from Jano van Hemert Engineering economic ramps

12. Going the last mile 12

13. Slide from Jano van Hemert

14. Gene Expression Run 1 1 2 3 4 5 6 7 C1 8 9 10 11 12 13 14 C2 Run 2 1 2 3 4 5 6 7 C3 8 9 10 11 12 13 14 C4 15 16 17 18 19 20 21 C5 22 23 24 25 26 27 28 C6 Run 3 15 16 17 18 19 20 21 C7 22 23 24 25 26 27 28 C8 29 31 32 33 34 35 36 C9 37 38 39 40 41 42 43 C10 Run 4 29 31 32 33 34 35 36 C11 37 38 39 40 41 42 43 C12 44 45 46 47 48 49 50 C13 51 52 53 54 56 57 58 C14 Run 5 44 45 46 47 48 49 50 C15 51 52 53 54 56 57 58 C16 55 59 60 61 62 63 3* C17 55 59 60 61 62 63 3* C18 1 2 3 4 5 6 7 C1 8 9 10 11 12 13 14 C2 1 2 3 4 5 6 7 C3 8 9 10 11 12 13 14 C4 15 16 17 18 19 20 21 C5 22 23 24 25 26 27 28 C6 15 16 17 18 19 20 21 C7 22 23 24 25 26 27 28 C8 29 31 32 33 34 35 36 C9 37 38 39 40 41 42 43 C10 29 31 32 33 34 35 36 C11 37 38 39 40 41 42 43 C12 44 45 46 47 48 49 50 C13 51 52 53 54 56 57 58 C14 44 45 46 47 48 49 50 C15 51 52 53 54 56 57 58 C16 55 59 60 61 62 63 3* C17 55 59 60 61 62 63 3* C18 Slide from Rob Kitchin

15. Walking a path together 15 co-shaping co-design co-creation co-constitution co-evolution co-construction co-  Finding a niche in the digital ecosystem

16. Alignment of paths to routine use Invention Proof of concept demonstration Local group use Filling a research niche Community use Established but still evolving Widespread and global use de facto standard 16 Competition for mind-share and resources

17. A data-intensive future e-Science Research Slide from Jano van Hemert

18. General Principles Support for research data should be in harmony with the evolving digital-data ecosystem Increase investment in analysing data to be commensurate with that for collecting data Co-evolve research practices with new methods and their supporting software Democratise research by improving education and access Smooth the path from foundational research, through invention and proof of concept to sustained use Expose the costs of computation and data to researchers 18

19. Recommendations Stimulate new thinking and international collaboration Invest and collaborate in creating shared methods and their supporting software for exploring and exploiting digital data Build intellectual ramps to new methods and provide convenient services for routine tasks Invest in the foundations for exploiting research data Develop a smooth path from method invention to their sustained and routine use 19

20. Actions 1.Workshops on DIR 2.DIR education 3.Ideas factory 4.Engage with current best practice 5.Immediate research challenges 6.DIR facilities 7.Boost reference data services 8.Foundational research 9.Green DIR 10.Coordination 20

21. Take home message Survival in the digital-data revolution depends on speed and appropriateness of adaptation 21

22. 22 ADMIRE – Framework 7 ICT 215024 ? Picture composition by Luke Humphry based on prior art by Frans Hals www.omii.ac.uk www.admire-project.eu www.ogsadai.org.uk www.nesc.ac.uk

23. Contact David De Roure dder@ecs.soton.ac.uk Carole Goble carole.goble@manchester.ac.uk Visit wiki.myexperiment.org

24. Logo store