Download presentation

Presentation is loading. Please wait.

Published byTimothy Murrell Modified about 1 year ago

1
Particle Swarm Optimization Fahimeh Fooladgar

2
2/48 Outline Swarm Intelligence Introduction to PSO Original PSO algorithms –Global Best PSO –Local Best PSO Algorithm Aspects Basic Variations PSO Parameters Application

3
3/48 Swarm Intelligence Example : benefits of cooperation Swarm group –agents that communicate with each other –either directly or indirectly –acting on their local environment Swarm Intelligence (SI) or collective intelligence –emerges from the interaction of such agents Computational Swarm Intelligence(CSI) – algorithmic models of such behavior

4
4/48 Swarm Intelligence(cont.) computational models of swarm intelligence – social animals and social insects –ants, termites, bees, spiders, fish schools, and bird flocks individuals relatively simple in structure but their collective behavior usually very complex –pattern of interactions between the individuals of the swarm over time

5
5/48 Swarm Intelligence(cont.) objective of computational swarm intelligence models –simple behaviors of individuals –local interactions with the environment and neighboring –to obtain more complex behaviors –solve complex problems (optimization problems)

6
6/48 Introduction First introduced by James Kennedy and Russell Eberhart in 1995 population-based search algorithm simulation of the social behavior of birds within a flock Individuals are particles Individuals follow a very simple behavior –emulate the success of neighboring –emulate their own successes

7
7/48 Introduction (cont.) swarm of particles : population of individuals particle have its own velocity x i (t): position of particle i at t

8
8/48 Introduction (cont.) velocity vector drives the optimization process reflects experiential knowledge and socially exchanged information The experiential knowledge of a particle: cognitive component –distance of the particle from its own best position –particle’s personal best position socially exchanged information :social component

9
9/48 Original PSO algorithms Two PSO algorithms –Differ in the size of their neighborhoods –gbest PSO and lbest PSO

10
10/48 Global Best PSO Neighborhood for each particle is entire swarm Social network : star topology Velocity update statement

11
11/48 Global Best PSO (cont.) v ij (t) :velocity of particle i in dimension j = 1,..., n x y ij (t) : personal best position y ^ j (t) : best position found by the swarm x ij (t) : position of particle i in dimension j c1 and c2 : positive acceleration constants –scale the contribution of the cognitive and social components r 1j (t), r 2j (t) ∼ U(0, 1) –stochastic element to the algorithm

12
12/48 Global Best PSO (cont.) fitness function personal best position at the next time step

13
13/48 Global Best PSO (cont.) global best position or n s : total number of particles in the swarm

14
14/48 Global Best PSO (cont.)

15
15/48 Local Best PSO smaller neighborhoods are defined for each particle network topology : ring social Velocity update statement

16
16/48 Local Best PSO(cont.) y ^ ij : best position, found by the neighborhood of particle i in dimension j best position found in the neighborhood N i

17
17/48 Local Best PSO(cont.) neighborhood defined gbest PSO is a special case of the lbest PSO with n Ni = n s

18
18/48 lbest PSO versus gbest PSO Two main differences –gbest PSO converges faster than lbest PSO less diversity –lbest PSO less susceptible to being trapped in local minima

19
19/48 Velocity Components v i (t) : previous velocity –memory of the previous flight direction –prevents the particle from drastically changing direction –bias towards the current direction –referred as the inertia component

20
20/48 Velocity Components(cont.) c 1 r 1 (y i −x i ) : cognitive component –drawn back particle to their own best positions, –individuals return to situations that satisfied them most in the past –referred to as the “nostalgia” of the particle

21
21/48 Velocity Components(cont.) social component In gbest PSO In lbest PSO each particle drawn towards the best position found by the particle’s neighborhood referred to as the “envy”

22
22/48 Geometric Illustration cognitive velocity inertia velocity social velocity new velocity

23
23/48 Algorithm Aspects initialize the swarm –Particle position –initial velocities –Initial personal best position

24
24/48 Stopping conditions Maximum number of iterations Acceptable solution has been found No improvement is observed over a number of iterations –if the average change in particle positions is small – if the average particle velocity over a number of iterations is approximately zero

25
25/48 Stopping conditions(cont.) Objective function slope is approximately zero If f ’(t) < Є,the swarm is converged

26
26/48 Social Network Structures StarRing Wheel Von NeumannFour Clusters Pyramid

27
27/48 Basic Variations Improve basic PSO –speed of convergence –Quality of solutions Velocity clamping Inertia weight Constriction Coefficient

28
28/48 Velocity Clamping exploration–exploitation trade-off Exploration : explore different regions of the search space Exploitation : concentrate the search around a promising area good optimization algorithm: balances these contradictory objectives –velocity update equation

29
29/48 Velocity Clamping(cont.) velocity quickly explodes to large values Then particles have large position updates –particles diverge Should control the global exploration of particles velocities clamped to stay within boundary constraints –V max,j denote the maximum allowed velocity in dimension j

30
30/48 Velocity Clamping(cont.) Large values of V max,j facilitate global exploration smaller values encourage local exploitation

31
31/48 Velocity Clamping(cont.) If V max,j is too small –swarm may not explore sufficiently beyond locally good regions –increase the number of time steps to reach an optimum –swarm may become trapped in a local optimum If V max,j is too large –risk the possibility of missing a good region –particles may jump over good solutions –but particles are moving faster

32
32/48 Velocity Clamping(cont.) Balance between –moving too fast or too slow –exploration and exploitation value of δ is problem-dependent

33
33/48 Inertia Weight introduced by Shi and Eberhart –control the exploration and exploitation abilities of the swarm –eliminate the need for velocity clamping controlling influence of previous flight direction to new velocity

34
34/48 Inertia Weight(cont.) value of w is extremely important –ensure convergent behavior –tradeoff exploration and exploitation For w ≥ 1 –velocities increase over time –the swarm diverges – Particles fail to change direction For w < 1 – particles decelerate until their velocities reach zero

35
35/48 Inertia Weight(cont.) guarantees convergent particle trajectories If this condition is not satisfied, divergent or cyclic behavior may occur

36
36/48 Inertia Weight(cont.) Dynamic Inertia Weight approaches –Linear decreasing Start with w(0)=0.9 and final inertia weight w(n t )=0.4 –n t : maximum number of time steps –w(0) is the initial inertia weight – w(n t ) is the final inertia weight –w(t) is the inertia at time step t

37
37/48 Inertia Weight(cont.) Random adjustments Nonlinear decreasing

38
38/48 Constriction Coefficient similar to the inertia weight –balance the exploration–exploitation trade-off –velocities are constricted by a constant χ – referred to as the constriction coefficient

39
39/48 Constriction Coefficient(cont.) Κ controls the exploration and exploitation For κ ≈ 0 –fast convergence –local exploitation For κ ≈ 1 –slow convergence –high degree of exploration Usually, κ set to a constant value First K set close to one, decreasing it to zero

40
40/48 Constriction Coefficient(cont.) Constriction approach equivalent to inertia weight approach if

41
41/48 PSO Parameters Swarm size (n s ) –more particles in the swarm, larger the initial diversity of the swarm –general heuristic : n s ∈ [10, 30] – actually problem dependent Neighborhood size –Smaller neighborhoods, slower in convergence, more reliable convergence to optimal solutions –Best solution : starting with small neighborhoods and increasing the neighborhood Number of iterations –It depend on problem

42
42/48 PSO Parameters(cont.) Acceleration coefficients – c 1, c 2, r 1 and r 2 – control the stochastic influence of the cognitive and social components –c 1 : how much confidence a particle in itself –c 2 : how much confidence a particle in its neighbors

43
43/48 PSO Application PercentPaper#Application Image processing Control Electronics and electromagnetics antenna design Power systems and plants Scheduling Design Communication networks Biological and medical Clustering and classification Fuzzy and neuro fuzzy Signal processing Neural networks Combinatorial optimization

44
44/48 PSO Application(cont.)

45
45/48 What makes PSO so attractive to practitioners? Simplicity Easy to implement –n s ×n x array for particle’s position –n s ×n x array particle’s velocity –n s ×n x d array particle’s personal best –1×n x array for global best –1×n x array for V max Can adapt to different application

46
46/48 What makes PSO so attractive to practitioners? All operations are simple and easy to implement It require low computational resources (Memory and CPU) It has ability to quickly converge to a reasonably good solution It can easily and effectively run in distributed environments

47
47/48 References A.P.Engelbrecht, “computational intelligence ”,2007 R.Poli, " Analysis of the Publications on the Applications of Particle Swarm Optimisation ", Journal of Artificial Evolution and Applications, Vol. 2008,10 pages, 2007 K.E. Parsopoulos and M.N. Vrahatis. Particle Swarm Optimizer in Noisy and Continuously Changing Environments. In Proceedings of the IASTED International Conference on Artificial Intelligence and Soft Computing, pages 289–294,2001

48
Thanks for your attention ? ?

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google