1. SAN DIEGO SUPERCOMPUTER CENTER Advanced User Support Project Overview Thomas E. Cheatham III University of Utah Jan 14th 2010 By Ross C. Walker

2. SAN DIEGO SUPERCOMPUTER CENTER Project Overview The Problem: Extensive work has been undertaken to improve the performance and scalability of Molecular Dynamics (MD) packages. NAMD AMBER GROMACS LAMMPS DESMOND

3. SAN DIEGO SUPERCOMPUTER CENTER Project Overview The Problem: Larger and Larger systems are being simulated. (1 million + atoms now routine) Longer and longer simulations are being run. (100ns to 1 microsecond now routine) Use of GPUs is leading to longer simulations being run on ‘basic’ desktops. This is leading to an explosion in trajectory size. The analysis tools are not keeping up with this.

4. SAN DIEGO SUPERCOMPUTER CENTER Project Overview Development of Parallel Analysis Tools for Molecular Dynamics Project composed of 3 sections: Development of parallel analysis tools for MD. Improvements in the handling of large datasets. Development of new analysis techniques for large and/or long timescale simulations.

5. SAN DIEGO SUPERCOMPUTER CENTER Phase 1 Parallelization of Ptraj Analysis Suite

6. SAN DIEGO SUPERCOMPUTER CENTER Ptraj Developed by Prof. Cheatham during the 1990’s at UCSF. Has been continually updated since. Distributed as openSource in the AMBERTools package. Supports a large range of analysis techniques on multiple file formats. (AMBER, CHARMM, NAMD etc…)

7. SAN DIEGO SUPERCOMPUTER CENTER Work to Date I have written a parallel version of ptraj that uses MPI calls and MPI/IO. It supports parallel I/O to take advantage of Lustre etc. Version 1.0 of the parallel version released at the end of December 2009 as part of free AMBERTools 1.3. (See http://ambermd.org/AmberTools-get.html)http://ambermd.org/AmberTools-get.html Multiple performance improvements also included. Random file access. Non-sequential access to frames in trajectory files etc.

8. SAN DIEGO SUPERCOMPUTER CENTER Version 1.0 Current supported parallel actions are: angle, atomicfluct, average, center, checkoverlap, closest, contacts,dihedrals, distance, image, principal, pucker, radgyr, randomizeions, rms,strip, translate, watershell For actions that are not yet parallel (e.g. hbond) the code will default back to single cpu operation.

9. SAN DIEGO SUPERCOMPUTER CENTER Example Performance

10. SAN DIEGO SUPERCOMPUTER CENTER Example Performance

11. SAN DIEGO SUPERCOMPUTER CENTER Random File Access Improvements The code can now seek randomly even for ASCII trajectory files. trajin foo.mdcrd 1001 1005 1 trajin foo2.mdcrd 23000 24000 100 image origin center trajout foobar.pdb pdb 1 TB trajectory file (Ranger) Original Code:5820 secs (97 mins) New Code:1 cpu37.5 secs 8 cpu4.8 secs

12. SAN DIEGO SUPERCOMPUTER CENTER Current Issues Cannot currently do random / parallel IO on compressed files. Code currently decompresses the file by calling gzip etc and streaming it. Suggestions? Compressed NetCDF allows this? Use of non-parallel action currently forces the entire run to be serial.

13. SAN DIEGO SUPERCOMPUTER CENTER Planned Work for the Coming Year Complete parallelization of remaining actions. Develop distributed memory versions of the memory hungry routines (e.g. hbond) to support larger systems. Begin developing (in collaboration with Prof Cheatham’s lab) new analysis techniques that are more useful when looking at microsecond+ trajectories or multi-million atom simulations.