Presentation is loading. Please wait.

Presentation is loading. Please wait.

Ivec.org Team Quokka: Australia’s First Foray into the Student Cluster Challenge Rebecca Hartman-Baker, Ph.D. Senior Supercomputing Specialist & SCC Team.

Similar presentations


Presentation on theme: "Ivec.org Team Quokka: Australia’s First Foray into the Student Cluster Challenge Rebecca Hartman-Baker, Ph.D. Senior Supercomputing Specialist & SCC Team."— Presentation transcript:

1 ivec.org Team Quokka: Australia’s First Foray into the Student Cluster Challenge Rebecca Hartman-Baker, Ph.D. Senior Supercomputing Specialist & SCC Team Coach Happy Cheeks, Ph.D. Supercomputing Quokka & Team Mascot ivec.org

2 Outline I.Student Cluster Competition II.Building & Training SCC Team III.Cluster Design IV.Science Applications V.Competition ivec.org

3 I. STUDENT CLUSTER COMPETITION iVEC Student Cluster Competition Training Team, 2013

4 ivec.org Student Cluster Competition: Introduction 48-hour non-stop computing showdown held at annual supercomputing conference Held in Denver, Colorado, USA in 2013 Teams of undergraduates design, build, run apps on cluster Power constraint ~3000W (standard track) $2500 equipment constraint (commodity track)

5 ivec.org SCC: Introduction Key cluster architecture rules: Machine must contain only publicly available components All components must be turned on at all times (low-watt idle okay) When running applications, must not exceed V power draw (~3120 W)

6 ivec.org SCC: Introduction Applications: HPCC + 3 known apps, 1 mystery app 2013 apps: Graphlab, WRF, Nemo5, + Flying Snakes (OpenFOAM) Teams judged on throughput of work provided by judges, plus interview to determine depth of understanding

7 ivec.org SCC: History First held at 2007 Supercomputing conference, SC07 Brainchild of Brent Gorda, now General Manager, Intel High- Performance Data Division (formerly Whamcloud) Now three Student Cluster Competitions/year: China (April) ISC (June) SC (November)

8 ivec.org SCC: iVEC’s Motivation Increase computational science literacy in WA Develop future users/employees Train professional workforce for local industry Exposure for iVEC in international HPC community It sounded like fun!

9 ivec.org II. BUILDING & TRAINING THE TEAM “Compiling,”

10 ivec.org Starting iVEC’s SCC Team Began by raising interest at iVEC partner universities Contacted iVEC directors at universities, got leads for whom to contact First interest came from ECU (3 students) Other unis followed

11 ivec.org Sponsorship for SCC Team SGI NVIDIA Allinea Rio Tinto

12 ivec.org Sponsorship for SCC Team Discussed hardware sponsorship with Cray & SGI SGI first to commit, hardware + travel money Solicited financial sponsorship from mining companies in WA Rio Tinto committed to sponsor 3 students Obtained software & hardware sponsorship from Allinea & NVIDIA

13 ivec.org Team Hardware Sponsorship Most important sponsorship to get SGI very enthusiastic about sponsoring team Put best person in Asia-Pacific region on project Todd helped team: Select machine architecture Determine software stack Set the machine up in Perth & at competition

14 ivec.org Team Hardware Sponsorship When team decided to use GPUs in cluster, NVIDIA loaned us 8 K20X GPUs Received free of charge through academic program (had to return after competition  )

15 ivec.org Team Travel Sponsorship Travel to competition very expensive Budget: $3000/student SGI committed enough for half of team Solicited support from mining companies in WA, successful with Rio Tinto

16 ivec.org Team Software Sponsorship I “won” license for Allinea software I asked Allinea to sponsor license for team instead Allinea provided license for MAP & DDT products MAP: simple profiling tool, very useful to novice users DDT: parallel debugger, intuitive GUI

17 ivec.org Team Composition Breakdown: 3 Computer Science/Games majors from ECU 2 Physics/Computer Engineering majors from UWA 1 Geophysics major from Curtin Each student assigned areas of expertise (1 primary, 2 secondary) At beginning of training, I facilitated students’ development of team norms (standards of behavior) that proved very effective No conflicts, no inappropriate behavior

18 ivec.org III. CLUSTER DESIGN “Crazy Straws,”

19 ivec.org Cluster Design Designing cluster, generally the following must be considered: Cost Space Utility Performance Power Consumption Cost

20 ivec.org Cluster Design Designing cluster, generally the following must be considered: Cost Space Utility Performance Power Consumption Cost

21 ivec.org Cluster Design Architecture choices: All CPU nodes All accelerator nodes Hybrid CPU/accelerator Accelerator choices: NVIDIA Tesla Intel Xeon Phi Combination (?)

22 ivec.org Cluster Architecture 2 Pyramid nodes 4 x NVIDIA K20X 2 x Intel Ivy Bridge 12-core 64 GB 8 Hollister nodes 2 x Intel Ivy Bridge 12-core 64 GB Infiniband interconnect Stay within power budget by running only GPUs or CPUs

23 ivec.org Cluster Architecture Chose CPU/GPU hybrid architecture For good LINPACK performance Potential accelerated mystery app Maximize flops per watt performance

24 ivec.org Cluster Software CentOS 6 Ceph filesystem Open-source Software GCC OpenMPI Numerical libraries FFTW PETSc NetCDF HDF5 Proprietary software Intel compiler/MKL Allinea DDT/MAP PGI compiler CUDA

25 ivec.org Cluster Software: Ceph Each node has > 1TB disk, need parallel filesystem Could use Lustre, however issues with losing data if one node fails Ceph: distributed object store and file system designed to provide excellent performance, reliability and scalability

26 ivec.org Cluster Software: Ceph Ceph object-storage system with traditional file-system interface with POSIX semantics Looks like regular filesystem Directly mounted in recent CentOS kernel Underneath, Ceph keeps several copies of files balanced across hosts Metadata server cluster can expand/contract to fit file system Rebalance dynamically to distribute data (weighted distribution if disks differ in size)

27 ivec.org IV. APPLICATIONS “TornadoGuard,” https://xkcd.com/937/https://xkcd.com/937/

28 ivec.org Applications High-Performance LINPACK Graphlab NEMO5 WRF Mystery Application – Flying Snakes!

29 ivec.org HIGH-PERFORMANCE LINPACK

30 ivec.org Linpack History Linear algebra library written in Fortran Benchmarking added in late 1980s to estimate calculation times Initial releases used fixed matrix sizes 100 and 1000 Arbitrary problem size support added in 1991 LAPACK replaced the Linpack library for linear algebra, however Linpack benchmarking tool still used today

31 ivec.org HPL Standard Released in 2000 re-written in C and optimized for parallel computing Uses MPI and BLAS The standard benchmark used measure supercomputer performance Used to determine the Top500 Also used for stress testing and maintenance analysis

32 ivec.org CUDA HPL CUDA-accelerated Linpack released by NVIDIA available on developer zone Uses the GPU instead of the CPU and limited to GPU memory Popularity gaining with GPU providing better flops/watt Standard for HPL runs in Student Cluster Competitions

33 ivec.org TF Year

34 ivec.org GRAPHLAB

35 ivec.org GraphLab Toolkit for graph algorithms Topic Modelling Graph Analytics Clustering Collaborative Filtering Graphical Models Computer Vision

36 ivec.org Graphlab Applications Page rank (e.g., Google) Image reconstruction Recommendation predictions (e.g., Netflix) Image stitching (e.g., panoramic photos)

37 ivec.org NEMO5

38 ivec.org NEMO5 Stands for NanoElectronics MOdeling Tools Free for academic use, not exactly open source Evolved to current form over 15 years Developed by Purdue University

39 ivec.org NEMO5 NEMO5 designed to model at the atomic scale Simulation of nanostructure properties: strain relaxation, phonon modes, electronic structure, self- consistent Schrodinger-Poisson calculations, and quantum transport E.g., modelling Quantum Dots

40 ivec.org WRF

41 ivec.org WRF Next-generation mesoscale numerical weather prediction system Used for both weather prediction and research forecasting, throughout the world

42 ivec.org MYSTERY APPLICATION: FLYING SNAKES!

43 ivec.org Mystery Application Unknown application, presented at competition To prepare, compiled and ran one new code each week during 2 nd semester Gained experience with different types of compiles (e.g., edit makefiles, make.inc, cmake, autoconf, etc.) Gained familiarity with common errors encountered while compiling, and how to fix them

44 ivec.org Flying Snakes! Aerodynamics of flying snakes Flying snakes inhabit rainforest canopy in East Asia & jump between tree branches, gliding to next branch Case of fluid dynamics: behavior of air as snake passes through, development of vortices, eddies, etc. Modeled with OpenFOAM, open-source computational fluid dynamics toolbox

45 ivec.org V. COMPETITION “Standards,”

46 ivec.org Competition Arrived in Denver Thursday before competition, to acclimate to 15-hour time difference Visited National Renewable Energy Laboratory to see supercomputers Began setting up on Saturday before competition Competition time: Monday evening – Wednesday evening Wednesday evening: party at Casa Bonita Thursday: Pros vs. amateurs competition Friday: back home

47 ivec.org Scenes from the Trip

48 ivec.org Scenes from the Trip

49 ivec.org Team Booth

50 ivec.org Casa Bonita

51 ivec.org Taking Down the Booth

52 ivec.org Results Official champion: University of Texas (last year’s champions too) Other rankings not given, but we were middle of pack Entire team (including coach) learned a lot! Students have potential leads for jobs & further study Plans to coach another team for 2014

53 ivec.org Bibliography CentOS, Ceph, High-Performance Linpack & HPCC, Graphlab, NEMO5, https://engineering.purdue.edu/gekcogrp/softwar e-projects/nemo5/ https://engineering.purdue.edu/gekcogrp/softwar e-projects/nemo5/ WRF, Krishnan et al., Lift and wakes of flying snakes, OpenFOAM,

54 ivec.org For More Information iVEC, Student Cluster Competition,


Download ppt "Ivec.org Team Quokka: Australia’s First Foray into the Student Cluster Challenge Rebecca Hartman-Baker, Ph.D. Senior Supercomputing Specialist & SCC Team."

Similar presentations


Ads by Google