Presentation is loading. Please wait.

Presentation is loading. Please wait.

Particle Swarm Optimization Fahimeh Fooladgar. 2/48 Outline Swarm Intelligence Introduction to PSO Original PSO algorithms –Global Best PSO –Local Best.

Similar presentations


Presentation on theme: "Particle Swarm Optimization Fahimeh Fooladgar. 2/48 Outline Swarm Intelligence Introduction to PSO Original PSO algorithms –Global Best PSO –Local Best."— Presentation transcript:

1 Particle Swarm Optimization Fahimeh Fooladgar

2 2/48 Outline Swarm Intelligence Introduction to PSO Original PSO algorithms –Global Best PSO –Local Best PSO Algorithm Aspects Basic Variations PSO Parameters Application

3 3/48 Swarm Intelligence Example : benefits of cooperation Swarm  group –agents that communicate with each other –either directly or indirectly –acting on their local environment Swarm Intelligence (SI) or collective intelligence –emerges from the interaction of such agents Computational Swarm Intelligence(CSI) – algorithmic models of such behavior

4 4/48 Swarm Intelligence(cont.) computational models of swarm intelligence – social animals and social insects –ants, termites, bees, spiders, fish schools, and bird flocks individuals relatively simple in structure but their collective behavior usually very complex –pattern of interactions between the individuals of the swarm over time

5 5/48 Swarm Intelligence(cont.) objective of computational swarm intelligence models –simple behaviors of individuals –local interactions with the environment and neighboring –to obtain more complex behaviors –solve complex problems (optimization problems)

6 6/48 Introduction First introduced by James Kennedy and Russell Eberhart in 1995 population-based search algorithm simulation of the social behavior of birds within a flock Individuals are particles Individuals follow a very simple behavior –emulate the success of neighboring –emulate their own successes

7 7/48 Introduction (cont.) swarm of particles : population of individuals particle have its own velocity x i (t): position of particle i at t

8 8/48 Introduction (cont.) velocity vector drives the optimization process reflects experiential knowledge and socially exchanged information The experiential knowledge of a particle: cognitive component –distance of the particle from its own best position –particle’s personal best position socially exchanged information :social component

9 9/48 Original PSO algorithms Two PSO algorithms –Differ in the size of their neighborhoods –gbest PSO and lbest PSO

10 10/48 Global Best PSO Neighborhood for each particle is entire swarm Social network : star topology Velocity update statement

11 11/48 Global Best PSO (cont.) v ij (t) :velocity of particle i in dimension j = 1,..., n x y ij (t) : personal best position y ^ j (t) : best position found by the swarm x ij (t) : position of particle i in dimension j c1 and c2 : positive acceleration constants –scale the contribution of the cognitive and social components r 1j (t), r 2j (t) ∼ U(0, 1) –stochastic element to the algorithm

12 12/48 Global Best PSO (cont.) fitness function personal best position at the next time step

13 13/48 Global Best PSO (cont.) global best position or n s : total number of particles in the swarm

14 14/48 Global Best PSO (cont.)

15 15/48 Local Best PSO smaller neighborhoods are defined for each particle network topology : ring social Velocity update statement

16 16/48 Local Best PSO(cont.) y ^ ij : best position, found by the neighborhood of particle i in dimension j best position found in the neighborhood N i

17 17/48 Local Best PSO(cont.) neighborhood defined gbest PSO is a special case of the lbest PSO with n Ni = n s

18 18/48 lbest PSO versus gbest PSO Two main differences –gbest PSO converges faster than lbest PSO  less diversity –lbest PSO less susceptible to being trapped in local minima

19 19/48 Velocity Components v i (t) : previous velocity –memory of the previous flight direction –prevents the particle from drastically changing direction –bias towards the current direction –referred as the inertia component

20 20/48 Velocity Components(cont.) c 1 r 1 (y i −x i ) : cognitive component –drawn back particle to their own best positions, –individuals return to situations that satisfied them most in the past –referred to as the “nostalgia” of the particle

21 21/48 Velocity Components(cont.) social component In gbest PSO In lbest PSO each particle drawn towards the best position found by the particle’s neighborhood referred to as the “envy”

22 22/48 Geometric Illustration cognitive velocity inertia velocity social velocity new velocity

23 23/48 Algorithm Aspects initialize the swarm –Particle position –initial velocities –Initial personal best position

24 24/48 Stopping conditions Maximum number of iterations Acceptable solution has been found No improvement is observed over a number of iterations –if the average change in particle positions is small – if the average particle velocity over a number of iterations is approximately zero

25 25/48 Stopping conditions(cont.) Objective function slope is approximately zero If f ’(t) < Є,the swarm is converged

26 26/48 Social Network Structures StarRing Wheel Von NeumannFour Clusters Pyramid

27 27/48 Basic Variations Improve basic PSO –speed of convergence –Quality of solutions Velocity clamping Inertia weight Constriction Coefficient

28 28/48 Velocity Clamping exploration–exploitation trade-off Exploration : explore different regions of the search space Exploitation : concentrate the search around a promising area good optimization algorithm: balances these contradictory objectives –velocity update equation

29 29/48 Velocity Clamping(cont.) velocity quickly explodes to large values Then particles have large position updates –particles diverge Should control the global exploration of particles velocities clamped to stay within boundary constraints –V max,j denote the maximum allowed velocity in dimension j

30 30/48 Velocity Clamping(cont.) Large values of V max,j facilitate global exploration smaller values encourage local exploitation

31 31/48 Velocity Clamping(cont.) If V max,j is too small –swarm may not explore sufficiently beyond locally good regions –increase the number of time steps to reach an optimum –swarm may become trapped in a local optimum If V max,j is too large –risk the possibility of missing a good region –particles may jump over good solutions –but particles are moving faster

32 32/48 Velocity Clamping(cont.) Balance between –moving too fast or too slow –exploration and exploitation value of δ is problem-dependent

33 33/48 Inertia Weight introduced by Shi and Eberhart –control the exploration and exploitation abilities of the swarm –eliminate the need for velocity clamping controlling influence of previous flight direction to new velocity

34 34/48 Inertia Weight(cont.) value of w is extremely important –ensure convergent behavior –tradeoff exploration and exploitation For w ≥ 1 –velocities increase over time –the swarm diverges – Particles fail to change direction For w < 1 – particles decelerate until their velocities reach zero

35 35/48 Inertia Weight(cont.) guarantees convergent particle trajectories If this condition is not satisfied, divergent or cyclic behavior may occur

36 36/48 Inertia Weight(cont.) Dynamic Inertia Weight approaches –Linear decreasing Start with w(0)=0.9 and final inertia weight w(n t )=0.4 –n t : maximum number of time steps –w(0) is the initial inertia weight – w(n t ) is the final inertia weight –w(t) is the inertia at time step t

37 37/48 Inertia Weight(cont.) Random adjustments Nonlinear decreasing

38 38/48 Constriction Coefficient similar to the inertia weight –balance the exploration–exploitation trade-off –velocities are constricted by a constant χ – referred to as the constriction coefficient

39 39/48 Constriction Coefficient(cont.) Κ controls the exploration and exploitation For κ ≈ 0 –fast convergence –local exploitation For κ ≈ 1 –slow convergence –high degree of exploration Usually, κ set to a constant value First K set close to one, decreasing it to zero

40 40/48 Constriction Coefficient(cont.) Constriction approach equivalent to inertia weight approach if

41 41/48 PSO Parameters Swarm size (n s ) –more particles in the swarm, larger the initial diversity of the swarm –general heuristic : n s ∈ [10, 30] – actually problem dependent Neighborhood size –Smaller neighborhoods, slower in convergence, more reliable convergence to optimal solutions –Best solution : starting with small neighborhoods and increasing the neighborhood Number of iterations –It depend on problem

42 42/48 PSO Parameters(cont.) Acceleration coefficients – c 1, c 2, r 1 and r 2 – control the stochastic influence of the cognitive and social components –c 1 : how much confidence a particle in itself –c 2 : how much confidence a particle in its neighbors

43 43/48 PSO Application PercentPaper#Application Image processing Control Electronics and electromagnetics antenna design Power systems and plants Scheduling Design Communication networks Biological and medical Clustering and classification Fuzzy and neuro fuzzy Signal processing Neural networks Combinatorial optimization

44 44/48 PSO Application(cont.)

45 45/48 What makes PSO so attractive to practitioners? Simplicity Easy to implement –n s ×n x array for particle’s position –n s ×n x array particle’s velocity –n s ×n x d array particle’s personal best –1×n x array for global best –1×n x array for V max Can adapt to different application

46 46/48 What makes PSO so attractive to practitioners? All operations are simple and easy to implement It require low computational resources (Memory and CPU) It has ability to quickly converge to a reasonably good solution It can easily and effectively run in distributed environments

47 47/48 References A.P.Engelbrecht, “computational intelligence ”,2007 R.Poli, " Analysis of the Publications on the Applications of Particle Swarm Optimisation ", Journal of Artificial Evolution and Applications, Vol. 2008,10 pages, 2007 K.E. Parsopoulos and M.N. Vrahatis. Particle Swarm Optimizer in Noisy and Continuously Changing Environments. In Proceedings of the IASTED International Conference on Artificial Intelligence and Soft Computing, pages 289–294,2001

48 Thanks for your attention ? ?


Download ppt "Particle Swarm Optimization Fahimeh Fooladgar. 2/48 Outline Swarm Intelligence Introduction to PSO Original PSO algorithms –Global Best PSO –Local Best."

Similar presentations


Ads by Google