Containers in HPC By Raja

Agenda: What is Containerization ? Different container Containers in HPC Why Singularity? Singularity Architecture Singularity Workflow Singularity commands State of Containers in HPC

What is Containerization? Containers in simple word: solution to the problem of how to get software to run reliably when moved from one environment to another. a container consists of an entire runtime environment: an application, plus all its dependencies, libraries and other binaries, and configuration files needed to run it, bundled into one package

Different Container: Docker Rkt: Rocket Java containers : standalone Java applications like Tomcat Unikernels LXD: From Ubuntu OpenVZ Windows Server Containers Hyper-V Containers

Traditional Virtualization & Modern Container: App2 Guest OS App3 App1 App4 App5 App6 Virtual Machines App1 Libs/Bins App2 App3 App4 Containers Container Engine Hypervisor Host OS Host OS Kernel Hardware Hardware

Containers in HPC: Docker DevOps, micro services, Enterprise applications Singularity : Lawrence Berkeley National Laboratory Application portability (single image file, contain all dependencies) Reproducibility, run cross platform, provide support for legacy OS and apps. Shifter : The National Energy Research Scientific Computing Center (NERSC) Utilize the large number of Docker apps. Provides a way to run them in HPC after a conversion process. It also strip out all the requirements of root so that they are runnable as user process. Univa Grid Engine (UGE) Running Docker containers in HPC, with UGE managing the Docker daemon process

Docker in HPC :Problem Statement Non-authorized users having root access to any of our production networks is considered a security breech Docker uses a root owned daemon No native GPU support No support for MPI… MPI developers estimate this milestone for 2 years from now! No Proper InfiniBand driver Doesn’t support production distributions (yet)… RHEL7 not even completely supported! Complexities with existing scheduling paradigms Root owned container daemons are outside the reach of the resource manager MPI/parallel job runs become increasingly complex

Why Singularity ? The Singularity is run by the user, loads the Singularity container and executes applications as the user. Shell environment is passed to the contained applications Singularity supports X11, MPI and resource managers natively. If you want to be root inside of the container, you must first be root outside of the container

Singularity Architecture:

Singularity Workflow:

Singularity General commands CONTAINER MANAGEMENT COMMANDS (requires root): help - Show additional help for a command bootstrap - Bootstrap a new Singularity image from scratch selftest - Run some self tests to make sure Singularity is installed and operating properly copy - Copy files from your host into the container create - Create a new container image CONTAINER USAGE COMMANDS: expand - Grow the container image exec - Execute a command within container Export - Export the contents of a container via a tar pipe run - Launch a runscript within container Shell - Run a Bourne shell within container Import - Import/add container contents via a tar pipe test - Execute any test code defined within container Mount - Mount a Singularity container image CONTAINER USAGE OPTIONS: CONTAINER REGISTRY COMMANDS: see singularity <command> help Pull - pull a Singularity Hub container to $PWD

State of Containers in HPC: What works well: New applications architected with containers in mind Sequential or multi-threaded Simple IO and networking dependencies What can work but with challenges: Distributed memory apps (like MPI) run on single host GPU integration (issues are socket affinity, for instance) What doesn't really work (yet): Cross-host distributed memory apps Choices Re-architect your software Look towards HPC-specific container technology (Shifter, Singularity) Wait for mainstream container innovation (like Docker) Focus adoption on new software

Demo on next session 