1. www.ci.anl.gov www.ci.uchicago.edu Accelerating data-intensive science by outsourcing the mundane Ian Foster

2. www.ci.anl.gov www.ci.uchicago.edu

3. The data deluge 1330 molec. bio databases Nucleic Acids Research (96 in Jan 2001) Genomic sequencing output x2 every 9 month >300 public centers 100,000 TB MACHO et al.: 1 TB Palomar: 3 TB 2MASS: 10 TB GALEX: 30 TB Sloan: 40 TB Pan-STARRS: 40,000 TB Climate model intercomparison project (CMIP) of the IPCC 2004: 36 TB 2012: 2,300 TB

4. www.ci.anl.gov www.ci.uchicago.edu 4 Big science has achieved big successes All build on NSF OCI (& DOE)-supported Globus Toolkit software LIGO: 1 PB data in last science run, distributed worldwide ESG: 1.2 PB climate data delivered to 23,000 users; 600+ pubs OSG: 1.4M CPU-hours/day, >90 sites, >3000 users, >260 pubs in 2010 Robust production solutions Substantial teams and expense Sustained, multi-year effort Application-specific solutions, built on common technology

5. www.ci.anl.gov www.ci.uchicago.edu 5 But small science is struggling More data, more complex data Ad-hoc solutions Inadequate software, hardware Data plan mandates

6. www.ci.anl.gov www.ci.uchicago.edu 6 Medium-scale science struggles too! Dark Energy Survey receives 100,000 files each night in Illinois They transmit files to Texas for analysis … then move results back to Illinois Process must be reliable, routine, and efficient The cyberinfrastructure team is not large Image credit: Roger Smith/NOAO/AURA/NSF Blanco 4m on Cerro Tololo

7. www.ci.anl.gov www.ci.uchicago.edu 7 The challenge of staying competitive "Well, in our country," said Alice … "you'd generally get to somewhere else — if you run very fast for a long time, as we've been doing.” "A slow sort of country!" said the Queen. "Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!"

8. www.ci.anl.gov www.ci.uchicago.edu 8 Current approaches are unsustainable Small laboratories – PI, postdoc, technician, grad students – Estimate 5,000 across US university community – Average ill-spent/unmet need of 0.5 FTE/lab? Medium-scale projects – Multiple PIs, a few software engineers – Estimate 500 across US university community – Average ill-spent/unmet need of 3 FTE/project? Total 4000 FTE: at ~$100K/FTE => $400M/yr Plus computers, storage, opportunity costs, …

9. www.ci.anl.gov www.ci.uchicago.edu 9 And don’t forget administrative costs 42% of the time spent by an average PI on a federally funded research project was reported to be expended on administrative tasks related to that project rather than on research — Federal Demonstration Partnership faculty burden survey, 2007

10. www.ci.anl.gov www.ci.uchicago.edu 10 You can run a company from a coffee shop

11. www.ci.anl.gov www.ci.uchicago.edu 11 Because businesses outsource their IT Web presence Email (hosted Exchange) Calendar Telephony (hosted VOIP) Human resources and payroll Accounting Customer relationship mgmt Software as a Service (SaaS)

12. www.ci.anl.gov www.ci.uchicago.edu 12 And often their large-scale computing too Web presence Email (hosted Exchange) Calendar Telephony (hosted VOIP) Human resources and payroll Accounting Customer relationship mgmt Data analytics Content distribution Infrastructure as a Service (IaaS) Software as a Service (SaaS)

13. www.ci.anl.gov www.ci.uchicago.edu 13 Let’s rethink how we provide research IT Accelerate discovery and innovation worldwide by providing research IT as a service Leverage software-as-a-service to provide millions of researchers with unprecedented access to powerful tools; enable a massive shortening of cycle times in time-consuming research processes; and reduce research IT costs dramatically via economies of scale

14. www.ci.anl.gov www.ci.uchicago.edu 14 Time-consuming tasks in science Run experiments Collect data Manage data Move data Acquire computers Analyze data Run simulations Compare experiment with simulation Search the literature Communicate with colleagues Publish papers Find, configure, install relevant software Find, access, analyze relevant data Order supplies Write proposals Write reports …

15. www.ci.anl.gov www.ci.uchicago.edu 15 Time-consuming tasks in science Run experiments Collect data Manage data Move data Acquire computers Analyze data Run simulations Compare experiment with simulation Search the literature Communicate with colleagues Publish papers Find, configure, install relevant software Find, access, analyze relevant data Order supplies Write proposals Write reports …

16. www.ci.anl.gov www.ci.uchicago.edu 16 A A B B Data movement can be surprisingly difficult

17. www.ci.anl.gov www.ci.uchicago.edu 17 A A B B Discover endpoints, determine available protocols, negotiate firewalls, configure software, manage space, determine required credentials, configure protocols, detect and respond to failures, determine expected performance, determine actual performance, identify diagnose and correct network misconfigurations, integrate with file systems, … Data movement can be surprisingly difficult It took 2 weeks and much help from many people to move 10 TB between California and Tennessee. (2007 BES report)

18. www.ci.anl.gov www.ci.uchicago.edu 18 Globus Online’s SaaS/Web 2.0 architecture Fire-and-forget data movement Automatic fault recovery High performance No client software install Across multiple security domains Web interface HTTP REST interface POST https://transfer.api. globusonline.org/ v0.10/ transfer Command line interface ls alcf#dtn:/ scp alcf#dtn:/myfile \ nersc#dtn:/myfile GridFTP servers FTP servers Other protocols: HTTP, WebDAV, SRM, … Globus Connect on local computers (Hosted on) (Operate) OpenID OAuth Shibboleth

19. www.ci.anl.gov www.ci.uchicago.edu 19 Example application: UC sequencing facility Sequencing instrument Mac using Globus Connect iBi File Server iBi general-purpose compute cluster Sequencing-specific compute cluster Mount drive Delivery of data to customer

20. www.ci.anl.gov www.ci.uchicago.edu 20 Statistics and user feedback Launched November 2010 >1700 users registered >500 TB user data moved >30 million user files moved >150 endpoints registered Widely used on TeraGrid/ XSEDE; other centers & facilities; internationally >20x faster than SCP Faster than hand-tuned “Last time I needed to fetch 100,000 files from NERSC, a graduate student babysat the process for a month.” “I expected to spend four weeks writing code to manage my data transfers; with Globus Online, I was up and running in five minutes.” “Transferred 28 MB in 20 minutes instead of 61 hours. Makes these global climate simulations manageable.”

21. www.ci.anl.gov www.ci.uchicago.edu 21 Moving 586 Terabytes in two weeks

22. www.ci.anl.gov www.ci.uchicago.edu 22 Monitoring provides deep visibility

23. Terabyte Gigabyte Megabyte Kilobyte 20 Terabytes in less than one day 20 Gigabyes in more than two days

24. www.ci.anl.gov www.ci.uchicago.edu 24 Common research data management steps Dark Energy Survey Galaxy genomics LIGO observatory SBGrid structural biology consortium NCAR climate data applications Land use change; economics

25. www.ci.anl.gov www.ci.uchicago.edu 25 We have choices of where to compute Campus systems – First target for many researchers XSEDE supercomputers – 220,000 cores, peer-reviewed awards – Optimized for scientific computing Open Science Grid – 60,000 cores; high throughput Commercial cloud providers – Instant access for small tasks – Expensive for big projects Users insist that they need everything connected

26. www.ci.anl.gov www.ci.uchicago.edu 26 Towards “research IT as a service”

27. www.ci.anl.gov www.ci.uchicago.edu 27 Research data management as a service GO-User – Credentials and other profile information GO-Transfer – Data movement GO-Team – Group membership GO-Collaborate – Connect to collaborative tools: Jira, Confluence, … GO-Store – Access to campus, cloud, XSEDE storage GO-Catalog – On-demand metadata catalogs GO-Compute – Access to computers GO-Galaxy – Share, create, run workflows Today Fall Prototype

28. www.ci.anl.gov www.ci.uchicago.edu 28 SaaS services in action: The XSEDE vision XUAS

29. www.ci.anl.gov www.ci.uchicago.edu 29 Data analysis as a service: Early steps Securely and reliably: 1.Assemble code 2.Find computers 3.Deploy code 4.Run program 5.Access data 6.Store data 7.Record workflow 8.Reuse workflow [3, 4] VM image App code Workflow Galaxy Condor Data store [5, 6] We have built such systems for biological, environmental, and economics researchers [1, 2] [7, 8]

30. www.ci.anl.gov www.ci.uchicago.edu 30 SaaS economics: A quick tutorial Lower per-user cost (x10?) via aggregation onto common infrastructure – $400M/yr  $40M/yr? Initial “cost trough” due to fixed costs Per-user revenue permits positive return to scale Further reduce per-user cost over time $ Time 0 X10 reduction in per-user cost: $50K  $5K/yr per lab $300K  $30K/yr per project

31. www.ci.anl.gov www.ci.uchicago.edu 31 A national cyberinfrastructure strategy? L L L L L L L L L L L L L L L L L L L L L L L L L L L P P P P Research data management Collaboration, computation Research administration Research data management Collaboration, computation Research administration To provide more capability for more people at less cost … Create infrastructure – Robust and universal – Economies of scale – Positive returns to scale Via the creative use of – Aggregation (“cloud”) – Federation (“grid”) Small and medium laboratories and projects a aaSaaS P

32. www.ci.anl.gov www.ci.uchicago.edu 32 Acknowledgments Colleagues at UChicago and Argonne – Steve Tuecke, Ravi Madduri, Kyle Chard, Tanu Malik, and others listed at www.globusonline.org/about/goteam/ Carl Kesselman and other colleagues at other institutions Participants in the recent ICiS workshop on “Human-Computer Symbiosis: 50 Years On” NSF OCI and MPS; DOE ASCR; and NIH for support

33. www.ci.anl.gov www.ci.uchicago.edu 33 For more information www.globusonline.org; @globusonline: Twitter www.globusonline.org Foster, I. Globus Online: Accelerating and democratizing science through cloud-based services. IEEE Internet Computing(May/June):70-73, 2011. Allen, B., Bresnahan, J., Childers, L., Foster, I., Kandaswamy, G., Kettimuthu, R., Kordas, J., Link, M., Martin, S., Pickett, K. and Tuecke, S. Globus Online: Radical Simplification of Data Movement via SaaS. Communications of the ACM, 2011.

34. www.ci.anl.gov www.ci.uchicago.edu Thank you! foster@uchicago.edu www.globusonline.org @globusonline