1. SWARM INTELLIGENCE IN DATA MINING Written by Crina Grosan, Ajith Abraham & Monica Chis Presented by Megan Rose Bryant

2. INTRODUCTION We will cover the following  Biological motivation of some theoretical concepts of swarm intelligence  Particle Swarm Optimization  Ant Colony Optimization  Basic Data Mining  Terminologies  Implementation with Swarm Intellegince Insect Swarm

3. BIOLOGICAL COLLECTIVE BEHAVIOR

4. Swarm behavior can be seen in bird flocks, fish schools, and insects. Group behaviors of some organisms are so integrated that they appear to move as a coherent entity. These behaviors are the influence behind swarm optimization. Swarm (“School”) of Fish

5. MAIN PRINCIPLES OF COLLECTIVE BEHAVIOR  Homogeneity: every organism in swarm has the same behavior model. No leader.  Locality: motion is influenced only by nearest members.  Collision Avoidance: avoids collision with nearby members.  Velocity Matching: attempt to match velocity of nearby members.  Flock Centering: attempt to stay close to nearby members.

6. TYPES OF COLLECTIVE DYNAMICAL BEHAVIOR  Swarm: an aggregate with cohesion, but low level of parallel alignment among members.  Torus: individuals perpetually rotate around an empty core. Direction of rotation is random.  Dynamic Parallel Group: individuals are polarized and move as a coherent group, but group form and density fluctuate.  Highly Parallel Group: much more static in terms of exchange of spatial positions. Form and density variety is minimal.

7. SWARMS AND ARTIFICIAL LIFE

8. Collective behavior algorithms have been applied to a variety of well-known algorithms including:  Traveling Salesman Problem  Quadratic Assignment Problem  Graph Problems  Clustering  Data mining  etc. TSP Point Set of Argentina

9. PARTICLE SWARM OPTIMIZATION (PSO) PSO is a population based search algorithm. Initialized with a population of random solutions (called ‘particles’). Each particle has an associated velocity. Particles fly through space with dynamically adjusted velocities according to historical behaviors. Particles fly towards better and better search area over time. Swarm of Starlings

10. BIOLOGICAL INTUITION FOR PSO Imagine the following: you are a bird in a flock of birds that is searching for a single French fry in a McDonald’s parking lot (the search area). You don’t know where the fry is, but you do know how far the food is and the position of all flock members. What is the best strategy to find the French fry? An effective strategy is to follow the bird closest to the food. Flock of Birds

11. PSO ALGORITHM PSO learns from this scenario and uses it to solve optimization problems. Each bird is a particle (a single solution). Each bird has a fitness value and a velocity. Birds fly through the search space by following the bird nearest the food thus far. Political Cartoon

12. ANT COLONIES OPTIMIZATION Now imagine that you are an ant among a colony of ants. When searching for food, you begin by searching the area closest to the nest in a random fashion. As you go, you leave behind a pheromone trail to tell your ant friends what you have found. When you find food, you use these pheromones to let everyone know how much there is and its quality. Pheromone Trail

13. DATA MINING

14. Data mining is the application of specific algorithms for extracting patterns from data. Historically, this application had been given many names including knowledge extraction, information discovery, and data pattern processing. Swarm Optimization can be very helpful in this process of Knowledge Discovery.

15. STEPS OF KNOWLEDGE DISCOVERY 1.Developing understanding of the domain, prior knowledge, and the goal. 2.Creating a target data set. 3.Data cleaning and preprocessing. 4.Data reduction and projection. 5.Matching the goals with a particular data mining method. Success requires continual growth

16. SWARM INTELLIGENCE AND KNOWLEDGE DISCOVERY

17. SWARM INTELLIGENCE & KNOWLEDGE DISCOVERY Data mining and swarm optimization can be used together to form a method that often leads to very good results. PSO methods have been used successful in pattern recognition, image processing, and unsupervised classification & image segmentation.

18. PSO IN DATA MINING Particle Swarm Optimization has been used in several data mining algorithms including the following:  Visual Data Mining  Recommender Systems  Classification Tasks  etc. PSO can often be employed when other implementations would be too large or too costly. Recommender System

19. ANT COLONY OPTIMIZATION AND DATA MINING

20. ANT COLONY OPTIMIZATION IN DATA MINING Ant Colony Optimization has been used with great success in clustering. Modeled after real ant behaviors, the computer ants ‘pick up’ data and move it to other areas with similar data. Several species have been studied to model different behaviors. Cluster Analysis

21. CONCLUSIONS

22. Biological behaviors can inform efficient optimization techniques such as Particle Swarm Optimization and Ant Colony Optimization. These optimization techniques have a variety of applications in Data Mining. They can often be employed when other techniques prove too costly. Zerg Rush, Starcraft II