1. Particle Swarm Optimization † Spencer Vogel † This presentation contains cheesy graphics and animations and they will be awesome

2. Particle Swarm Basics

3. ✘ Each particle is trying to find the global optimum ✘ Each particle is moving ✘ Each particle remembers where it’s local optima was Basic Idea

4. ✘ Each particle in the swarm cooperates with all of the other particles  Each particle has a neighborhood associated with it Neighborhoods

5. Social Geographical

6. ✘ Each particle in the swarm cooperates with all of the other particles  Each particle has a neighborhood associated with it  Each particle knows the fitness of all other particles in it’s neighborhood ҂ The best position from it’s neighborhood is used to adjust the particle’s velocity Neighborhoods

7. ✘ As each particle has to move, it has to move to a new position at each time step  It does this by adjusting it’s velocity  It’s velocity is based off a random weight of: ҂ It’s current velocity ҂ A random portion in the direction of it’s personal optimal fitness ҂ A random portion of the direction of the neighborhood optimal fitness Particle Action

8. Swarm Dynamics Current Motion Influence Swarm Influence Particle Memory Influence Resulting Vector Projected Motion

9. Common Test Functions Griewank Rastrigin Rosenbrock Sinenvsin

10. Parallelization

11. ✘ Distributes all data to be processed amongst all processors ✘ Allows individual processors to complete all tasks for one set of data ✘ Applied widely to genetic algorithms Data Parallelism

12. ✘ All particles are sent to the environment and the algorithm waits for all analysis to complete before continuing ✘ Usually results in poor efficiency as you end up waiting for some tasks to complete before updating the particles Synchronous

13. Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current State

14. ✘ Separates updates actions for each point and the swarm ✘ Updates point actions after each point is analyzed, and swarm actions at the end of each iteration Asynchronous

15. Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current State

16. ✘ Master  Initialize all optimization parameters, positions, and velocities  Holds a queue of particles for slave processors to evaluate  Updates global positions and velocities  Sends the position of the next particle in the que to an available slave processor  Receives cost function values from slave processors  Checks convergence Asynchronous master/slave paradigm

17. ✘ Slave  Receives particle position from master processor  Evaluates the fitness function at the given particle  Updates particle position  Updates particle velocity  Sends a cost function to the master processor Asynchronous master/slave paradigm

18. Asynchronous vs Synchronous (a) Asynchronous Implementation (b) Synchronous Implementation

19. ✘ H(x) represent different fitness functions ✘ Values are the mean (standard deviation) of function evaluations required to reach the final solution Performance Comparison

20. Efficiency Comparison

22. Thread communication Loosely correlated parameters Strongly correlated parameters

23. Thread Communication Unknown parameter correlation

24. Performance Comparison Communication style 1 with Rosenbrok Communication style 1 with Rastrigin Communication style 2 with Griewank Communication style 3

25. My Project

26. ✘ Written in C++ using OpenGL and boost  Uses a combination of glew and freeglut ✘ Allows variable initialization via a configuration file ✘ Particle’s “best fitness” value decreases over time ✘ Provides visual feedback of particles in the search space My Implementation

27. ✘ Parameter tuning ✘ Getting social neighborhoods to set up correctly ✘ The swarm currently has moments of “convergence” followed by moments of “oscillation”  Bug not currently fixed Challenges

28. Things to still implement

29. ✘ All points seek out the green dot (user controllable via mouse) ✘ 3 axis search ✘ Individual particle “depth” represented by red – purple – blue dots  Red represents + z axis  Blue represents – z-axis  Purple shades represent a neutral z axis Live Demo

30. Questions?

31. ✘ Chu, Shu-Chuan, John F. Roddick, and Jeng-Shyang Pan. "A Parallel Particle Swarm Optimization Algorithm With Communication Strategies." Harbin Institute of Technology. 13 Sept. 2002. Web. 15 Oct. 2015. ✘ Venter, Gerhard, and Jaroslaw Sobieszczanski-Sobiesk. "A Parallel Particle Swarm Optimization Algorithm Accelerated by Asynchronous Evaluations." EngOpt. 30 May 2005. Web. 10 Nov. 2015. ✘ Pinto, Andry, Hugo Alves, Inês Domingues, Luís Rocha, and Susana Cruz. "The Particle Swarm Optimization Algorithm." University of Porto. 2011. Web. 15 Oct. 2015 ✘ "Particle Swarm Optimization Demo." YouTube. YouTube, 8 Feb. 2012. Web. 15 Oct. 2015. References