Advanced User Support Amit Majumdar 8/13/09

Outline  Three categories of AUS  Operational Activities  AUS.ASTA  AUS.ASP  ASTA example

Three Categories of AUS I.Advanced Support for TeraGrid Applications (AUS.ASTA) I.AUS staff work on a particular user’s code (some community codes) II.Guided/initiated by the allocations process II.Advanced Support for Projects (AUS.ASP) I.Project initiated by AUS staff (jointly with users) – impacts many users II.Optimally install software, maintain, train users III.Advanced Support for EOT (AUS.ASEOT) I.Advanced HPC/CI trainings II.Participate in workshops, tutorials (TG09, PetaScale, SCXY) III.Interaction with other CI programs PlantCI, DataNet etc.

Update on Operational Activities  Bi-weekly telecon among AUS POCs from every RP site o Matching of AUS staff to ASTA projects, discussion about ASP, EOT activities o Program plans, annual reports etc.  Web/Tele-con among AUS technical staff o Biweekly technical tele/web-conference on ASTA and other projects - using readytalk o Since we started (early October, 2008) 20 tele/web-con sessions and 31 technical presentations (including today)

ASTAs Started in October TRAC #PI nameFieldAUS staff 1Jordan/USCGeo sciences Cui (SDSC ), Urbanic (PSC ), Kim(TACC ), Wong (NICS), Bock (NCSA) 2Mahesh/U. MinnCFDTatineni (SDSC), Crosby (NICS),Cazes (TACC ) 3Roitberg/U. FloridaBiochemWalker (SDSC ), Pamidighantam (NCSA ), Halloy (NICS ) 4Voth/U. UtahBioechemBlood (PSC ), Crosby (NICS), Peterson (TACC ) 5Buehler/MITCivil EngrWalker (SDSC) 6Durisen/IUAstroHenschel, Berry (IU) 7Papavassiliou/ OUCFDReddy (PSC ) 8Van Einde/UCSDStr. EngrChoi, Sivagnanam (SDSC), Halloy (NICS), Cazes (TACC)

Startup/Supplemental ASTAs #PI NameFieldAUS Staff 1Schulten/UIUC (completed)Bio ChemKoesterke, Liu, Milfeld, TACC 2Ferrante/U. WashingtonCFD/DNS/turbulenceTaha (NCSA), Pekurovsky (SDSC,), Bock (NCSA), Peterson (TACC), Crosby (NICS) 3Tabakin/U. PittQuantum ComputingGomez ( PSC) 4Mashayek/UICCFDTaha (NCSA) 5Malamataris/GMCFDBrook (NICS) 6Hu/CSULBPhysicsBarth (TACC) 7Yeung/GtechCFD/DNSWong(NICS), Pekurovsky (SDSC) 8Finol/CMUCFDJana (PSC) 9Geoffrey/CMUCSJana (PSC) 10Roberts/RTI;Karniadakis/Bro wn;Coveney/UCL CS/CFD/ChemistryKaronis (NIU, ANL) 11Kanamitsu/SIO-UCSDClimate SciencePeterson (TACC) 12Burke/Pitt (Completed)MedicineBrown (PSC)

ASTAs Started in January TRAC #PI NameFieldAUS staff 1 Aidun/GTCFDDillman (Purdue), Khurram (NCSA) 2Cheatham/U.UtahBiochemPamidighantam (NCSA), Crosby (NICS), Milfeld (TACC), Madrid (PSC), Walker (SDSC) 3Gygi/UCDMaterials ScienceFahey (NICS), Kim (TACC) 4Jansen/RPICFDWong (NICS), Peterson (TACC), O’neal (PSC) 5Van de Walle/UCSB (completed) Materials ScienceLiu (TACC), Vanomoer (NCSA) 6Scheraga/CornellChemistryBlood, Mahmoodi (PSC) 7Oleynik/U. South Florida Materials ScienceBarth (TACC), Crosby (NICS), Jundt (LONI)

ASTAs Started in April TRAC #PI NameFieldAUS staff 1Batista/LANLMaterials ScienceLiu (TACC) 2Schnetter/LSUAstrophysicsMahmoodi (PSC), Kufrin, Liu (NCSA), Pfeiffer (SDSC) 3Helly/SIO-UCSDClimate ScienceCheeseman (Purdue), Jundt (LONI) 4Liu/UKYQCDReddy (PSC), Fahey (NICS) 5Smith/GtechCFDTBD upon PI’s response 6Wheeler/UTCFDKhamra (TACC) 7Cen/PrincetonAstrophysicsBrook (NICS), Kharma(TACC), Chourasia (SDSC) 8Latour/ClemsonBioChemTBD upon PI’s response 9Apte/Oregon St.CFDRosales (TACC) 10Sandholm/CPUCSWelling (PSC)

ASTAs Started in July 2009 TRAC #PI NameFieldAUS staff 1Karimabadi/UCSDSpace PhysicsMahidhar (SDSC) 2Toomre/UColoradoAstrophysicsReddy (PSC), Cazws (TACC), Crosby (NICS) 3Miller/UCSDBioinformaticsPfeiffer (SDSC) 4Joshi/GTMech EngrKim (TACC), Cheeseman (Purdue), Mahmoodi (PSC) 5Wang/UT SWMolecular DynamicsPamidighantam (NCSA ) 6Khokhlov/UChicagoAstrophysics/visualizationKelly, Paul (TACC) 7Cobb/ORNLNeutron ScienceLynch, Chen (ORNL)

ASTAs Total ASTAs: 44 Two more Startup/Supplemental ASTAs approved 1Gordon/CMUKnowledge Models and Cognitive Systems Jana (PSC) 2Ostoja-Starzewski /UIUC CFDKoric (NCSA) New Supplemental ASTAs Since last Round Table

Advanced Support Projects Three projects (two completed first phase) – Benchmarking Molecular Dynamics codes (SDSC, PSC) – completed first phase – Benchmarking Materials Science codes (PSC, TACC, LONI, NCSA, NICS) – completed first phase – Hybrid Programming model for multi-core (TACC, SDSC, PSC, NICS) jointly with XS-WG - continuing

ASP Benchmark Webpage Doc group has helped in setting up of AUS benchmark pages Moved from earlier staging area webpage to production doc page:

ASTA Example: As a part of Adrian Roitberg ASTA GPU Implementation of AMBER by Ross Walker (SDSC) jointly with NVIDIA experts Other ASTA team members Sudhakar Pamidighantam (NCSA ), Christian Halloy (NICS )

GPU Support Collaboration with NVIDIA to produce CUDA version of AMBER. – PMEMD Engine – Implicit Solvent GB (V1.0 complete) – Explicit Solvent PME (in progress) Focus on accuracy. – It MUST pass the AMBER regression tests. – Energy conservation MUST be comparable to double precision CPU code.

GPU Accuracy Use of double precision in all places severely limits performance. – Make careful use of double precision where needed. – Calculate in single precision. – Accumulate in double precision. – Avoid large dynamic ranges in arithmetic expressions. – Switch over to double precision automatically if the dynamic range is too high.

Provisional Performance 1 x Tesla C1060 (NCSA Lincoln S1070 – 4X) Timings System Wall Time (s) NS/day ACE-ALA3-NME(GPU) ACE-ALA3-NME(4xCPU)* (*Note 4 cpu since we need 10x > atoms than processors) TRPCage(GPU) TRPCage(8xCPU) Myoglobin(GPU) Myoglobin(8xCPU) Nucleosome(GPU) Nucleosome(8xCPU) Cray XT5 (1024xCPU)