Transforming Science Through Data-driven Discovery Tools and Services Workshop Atmosphere Joslynn Lee – Data Science Educator Cold Spring Harbor Laboratory, DNA Learning Center

Welcome to Atmosphere Custom Cloud Computing for Life Sciences

What is Cloud Computing? Yet another round of jargon In the simplest terms, cloud computing means storing and accessing data and programs over the Internet instead of your computer's hard drive

Why Cloud Computing? Yet another round of jargon Some biological research problems require intense computation to those requiring little computation. Size of data sets will vary from MB to GB to TB. Advantageous to have a shared high performance computing (HPC) cluster and storage resources! BigDog at SIU! Atmosphere reduces the extensive time, resources, and overhead needed to set up analyses. Utilize virtual machines.

What is Cloud Computing? Important concept: Image: a template of a virtual machine containing an installed OS, software, configuration Image (file) Document(s) (file) Original system Local storage New “clone” system (files/data) Copied Document(s) (file) New system

Working with ‘Big’ Data Important concept: Instance – launched image of a virtual machine CyVerse Cloud (Disk + CPU + Memory) + (Image) Atmosphere Instance (virtual machine)

Working with ‘Big’ Data Important concept: Instance – launched image of a virtual machine CyVerse Cloud (Disk + CPU + Memory) + (Image) Atmosphere Instance (virtual machine) Anything that you would normally be able to do with your local laptop/desktop, you can do on a virtual machine in the Atmosphere.

Atmosphere Overview Largest, easiest to use for Life Sciences Choose an existing image or customize Instances up to 16-Core / 128 GB RAM Access via iCommands (shell) or VNC Share you image with selected users, or make them public

Atmosphere Overview Connecting to your instance VNC Viewer: PuTTy: WindowsMacLinux VNC Viewer Shell/terminal VNC Viewer Shell/terminal VNC Viewer PuTTY

Atmosphere Overview Connecting to your instance Work in an on-demand Linux environment (most bioinformatics) Collaborate with students and colleagues on the same instance Get Science Done Reproducibility Productivity Multicore high memory images to run multithreading applications Move your analyses from your laptop to the cloud Make data, workflows, and analyses available in a public image Access previous software version and images

Hands-on Demo Workshop packet: Atmosphere Cloud Computing Page 20 Handout the usernames / passwords

Atmosphere Overview Hands-on demo: Atmosphere Cloud Computing Launch and connect to Atmosphere – page 23 Connect to your Instance– page 23 Connect via VNC– page 24 By the end of this demo, you should be able to: Select and launch an instance Connect your instance to the Data Store Use an application in Atmosphere Understand how to pause, stop and terminate instances

Atmosphere Overview Login to Atmosphere  scroll down to icons Sign in on the top right corner

Atmosphere Overview Hands-on demo: Atmosphere Cloud Computing Launch and connect to Atmosphere – page 23 Connect to your Instance– page 23 Connect via VNC– page 24 By the end of this demo, you should be able to: Select and launch an instance Connect your instance to the Data Store Use an application in Atmosphere Understand how to pause, stop and terminate instances

Atmosphere Overview Key things to remember when you try this yourself Images do not have automatic access to your Data Store Use Cyberduck to access the Data Store Use iCommands Users have monthly allocation limits Terminate or stop instances not in use If a larger allocation is needed, contact support All data on terminated instances will be destroyed Use Cyberduck or iCommands to transfer data off the instance You may also create an EBS Volume (see documentation)

Atmosphere Overview User perspectives and possible applications Learned how to use the shell and how to work with Linux Mastered using R to develop plots for his manuscript Launches an image and has full SUDO access to customize Developed a software with numerous R and Python library dependencies She updates it regularly by making a new image Linked several atmosphere instances with Apache Hadoop Worked with CyVerse to import existing Amazon image Bioinformatician Core Facilities Bench Scientist

Help: ask.iplantcollaborative.org Detailed instructions with videos, manuals, documentation in CyVerse Wiki Search by tag

Parker Antin Nirav Merchant Eric Lyons Matt Vaughn Doreen Ware Dave Micklos CyVerse is supported by the National Science Foundation under Grant No. DBI and DBI Executive Team Transforming Science Through Data-driven Discovery