10/21/20091 Protein Explorer: A Petaflops Special-Purpose Computer System for Molecular Dynamics Simulations Makoto Taiji, Tetsu Narumi, Yousuke Ohno, Noriyuki Futatsugi, Atsushi Suenaga, Naoki Takada and Akihiko Konagya High-Performance Biocomputing Research Team, Bioinformatics Group, Genomic Sciences Center, Institute of Physical and Chemical Research (RIKEN) ACM/IEEE SC2003 Conference Presented by Jing Xue For Computer Architecture Reading Group

10/21/20092 Protein Explorer (PE) System What is PE system ? –special-purpose computer system for molecular dynamics simulations particularly protein simulations –PC cluster equipped with special-purpose engines that calculate non-bonded interactions between atoms What is MDGRAPE-3 Chip? –A dedicated LSI performs these force calculations Why is specialized? –Most time is spent on the calculation of long-range forces Gravitational, Coulomb, and Van der Waals forces –Communication-to-computation ratio is 0.25 Gbytes/sec*Tflops What is GRAPE? –GRAPE (GRAvity PipE) is a project to develop high-performance competitive special purpose systems –Classical particles (e.g. gravitational N-body problems, Molecular dynamics (MD) simulations) What is pioneered? –Delft Molecular Dynamics Processor (DMDP) and FASTRUN processor What is the targets of PE system? –High-precision screening for drug design and large scale simulations –Funded by Protein 3000 project started in 2002

10/21/20093 MDGRAPE-3 System

10/21/20094 What Protein Explorer Calculates Two-body forces on i-th particle Fi Where,. The vectors ri, rj are the position vectors of the i,j-th particles and is an arbitrary smooth function. Coulomb forces: Lennard-Jones potential:

10/21/20095 MDGRAPE-3 chip

10/21/20096 Software Cost and Performance Estimation 1.Call pe_start_calc 2.Perform other force calculations 3.Call pe_wait_calc (force) 4.Add other forces such as bonding force to ‘force’, then calculates orbits of particles, and increments a time 1.Get command sequence from host memory 2.Write positions, charges, and species of j- particles to the chip 3.Broadcast positions of 40 i-particles to all the chip 4.Issue the calculation command 5.Do calculation 6.Read results out from the chip 7.Transfer result to host memory 8.Repeat from 3-7 for all i-particles 9.Tell the end of the calculation to the host

10/21/20097 Anton, a Special-Purpose Machine for Molecular Dynamics Simulation David E. Shaw et. al. D. E. Shaw Research ISCA 2007 Presented by Alok Garg

10/21/20098 Force Calculation Mechanics force fields on 200,000 molecules/time step Total energy –Bonded (linear time complexity) –Non-bonded - O(n 2 ) Non-bonded interactions –Range limited interactions –Long-range interactions Expresses as a convolution Computed by: FFT, multiplication, and inverse FFT 73% computation is range limited interactions

10/21/20099 Range-Limited Pair-wise Interactions

10/21/ Parallelization Algorithm

10/21/ Computation e.g. A cubical system of 25,000 atoms Atoms in tower: 220 Atoms in plate: 430 Total pairs: 94,600 23,000 satisfy the selection rule Each node must import 550 particles

10/21/ Specialized Hardware

10/21/ Rest of the Machine

10/21/ Major Computational Tasks

10/21/ System Architecture Features –Multicast support –Compressed transfer of sparse data structures Push based communication Memory accumulate forces for each particle

10/21/ Performance Results