1 Evaluation of parallel particle swarm optimization algorithms within the CUDA™ architecture Luca Mussi, Fabio Daolio, Stefano Cagnoni, Information Sciences, 181(20), 2011, pp Presenter: Guan-Yu Chen Stu. No. : MA0G0202 Advisor : Shu-Chen Cheng

2 Outline 1.Particle swarm optimization (PSO) 2.PSO parallelization 3.The CUDA™ architecture 4.Parallel PSO within the CUDA™ 5.Results 6.Final remarks

3 1. Particle swarm optimization (1/3) Kennedy & Eberhart (1995). –Velocity function. –Fitness function.

4 1. Particle swarm optimization (2/3) Velocity function V: the velocity of a particle. X gbest : the best-fitness point ever found by the whole swarm. C 1, C 2 : two positive constants. R 1, R 2 : two random numbers uniformly drawn between 0 and 1. w: inertia weight. X: the position of a particle. X lbest : the best-fitness position reached by the particle. t: at time t.

5 1. Particle swarm optimization (3/3) Fitness function  Self-definition.

6 2. PSO parallelization Master-Slave paradigm. Island model (coarse-grained algorithms). Cellular model (fine-grained paradigm). Synchronous or Asynchronous.

7 3. The CUDA™ architecture (1/5) CUDA™ (nVIDIA™, Nov. 2006). –A handy tool to develop scientific programs oriented to massively parallel computation. Kernels  Grid  Thread blocks  Threads How many thread blocks for problem ？ How many threads per thread block?

8 3. The CUDA™ architecture (2/5) Streaming Multiprocessors (SMs) –8 scalar processing cores, –A number of fast 32-bit registers, –A parallel data cache shared between all cores, –A read-only constant cache, –A read-only texture cache.

9 3. The CUDA™ architecture (3/5) SIMT (Single Instruction, Multiple Thread) –Which creates, manages, schedules, and executes groups (warps) of 32 parallel threads. –The main difference from a SIMD (Single Instruction, Multiple Data) architecture is that SIMT instructions specify the whole execution and branching behavior of a single thread.

10 3. The CUDA™ architecture (4/5) Each kernel should reflect the following structure: a)Load data from global/texture memory; b)Process data; c)Store results back to global memory.

11 3. The CUDA™ architecture (5/5) The most important specific programming guidelines: a)Minimize data transfers between the host and the graphics card; b)Minimize the use of global memory: shared memory should be preferred; c)Ensure global memory accesses are coalesced whenever possible; d)Avoid different execution paths within the same warp.

12 4. Parallel PSO within the CUDA™ The main obstacle to PSO parallelization is the dependence between particle’s updates.  SyncPSO –X gbest or X lbest are updated at the end of each generation only.  RingPSO –Relaxing the synchronization constraint. –Allowing the computation load to be distributed over all SMs available.

Basic parallel PSO design (1/2)

Basic parallel PSO design (2/2)

Multi-kernel parallel PSO algorithm (1/3) posID = ( swarmID * n + particleID ) * D + dimensionID

Multi-kernel parallel PSO algorithm (2/3) PositionUpdateKernel ( 1st kernel ) –Be used to update the particles’ positions by scheduling a number of thread blocks equal to the number of particles. FitnessKernel ( 2nd kernel ) –Be used to compute the fitness. BestUpdateKernel ( 3rd kernel ) –Be used to update X gbest or X lbest.

Multi-kernel parallel PSO algorithm (3/3)

18 5. Results w = and C 1 = C 2 =

SyncPSO (1/2) Asus GeForce EN8800GT GPU; Intel Core2 Duo™ CPU 1.86 GHz. a)100 consecutive runs; a single swarm of 32, 64, and 128 particles; 5-dimensional Rastrigin function vs. the number of generations. b)run 10,000 generations of one swarm with 32, 64 and 128 particles scales with respect to the dimension of the generalized Rastrigin function (up to nine dimensions).

SyncPSO (2/2)

RingPSO (1/5) nVIDIA™ Quadro FX 5800; Zotac GeForce GTX260 AMP 2 edition; Asus GeForce EN8800GT SPSO on 64-bit Intel(R) Core(TM) i7 CPU 2.67 GHz. 1)The sequential SPSO version modified to implement the ring topology; 2)The ‘basic’ three-kernel version of RingPSO; 3)RingPSO implemented with two kernels only (one kernel which fuses BestUpdateKernel and PositionUpdateKernel, and FitnessKernel)

RingPSO (2/5) Sphere function D [100, 100]

RingPSO (3/5) Rastrigin function D[5.12, 5.12 ]

RingPSO (4/5) Rosenbrock function D[30, 30]

RingPSO (5/5)

26 6. Final remarks (1/2) SyncPSO is usually more than enough for any practical application. SyncPSO’s usage of computation resources is very inefficient in cases when only one or few swarms need to be simulated. SyncPSO becomes inefficient when the problem size increases above a certain threshold.

27 6. Final remarks (2/2) The drawbacks of accessing global memory for the multi-kernel version are more than compensated by the advantages of parallelization. The speed-up for the multi-kernel version increases with problem size. Both versions are far better than the most recent results published on the same task.

28 The End~ Thanks for your attention!!