Presentation is loading. Please wait.

Presentation is loading. Please wait.

An introduction to search and optimisation using Stochastic Diffusion Processes Stochastic Diffusion Processes define a family of agent based search and.

Similar presentations

Presentation on theme: "An introduction to search and optimisation using Stochastic Diffusion Processes Stochastic Diffusion Processes define a family of agent based search and."— Presentation transcript:

1 An introduction to search and optimisation using Stochastic Diffusion Processes Stochastic Diffusion Processes define a family of agent based search and optimisation algorithms which have been successfully used in a variety of real-world applications and for which there is a sound theoretical foundation. - Mark Bishop Goldsmiths, University of London This presentation summarises recent research carried out by the Goldsmiths/Reading/Kings SDP group: Mark Bishop, Slawomir Nasuto, Kris de Meyer, Darren Myatt & Mohammad Majid. SDP resource pages are maintained at:

2 Some search and optimisation applications employing Stochastic Diffusion Processes Eye tracking Bishop & Torr. Lip tracking Grech-Cini & McKee. Mobile robot localisation Beattie et al. Site selection for wireless networks Hurley & Whitaker. Speech recognition Nicolaou. 3D computer vision Myatt et al. Models of attention Summers. A new connectionist paradigm for cognitive science Nasuto, Bishop et al. Theoretical Nasuto & Bishop. Sequence detection Jones.

3 The Restaurant Game: a simple Stochastic Diffusion optimisation A group of conference delegates arrive in a foreign town and want to find a good place to eat: the search space is the set of all restaurants; the objective function – restaurant quality - is the sum of numerous independent partial objective function evaluations; A random independent partial objective function evaluation is defined by a diners response to a randomly chosen meal: {GOOD or BAD}. Usually in a large town a naive exhaustive search will be impractical as there will be too many (restaurant dish) combinations to evaluate during the period the delegates are attending the conference.

4 Restaurant quality: a stochastic dynamic objective function The objective function optimised by the Restaurant Game - restaurant quality - is a stochastic variable defined by the sum of mean diner responses to all the meal combinations offered by a given restaurant. Restaurant quality is a stochastic variable as the perceived quality of each meal may vary: each time a meal is prepared; with changes in a diners mood; as each diner is likely to have a different perception of what tastes good. In other best-fit searches the partial evaluation of the objective function is typically deterministically defined. E.g. Is a specific coloured-feature red? Is a specific numeric-feature = 10, etc.

5 Stochastic Diffusion Search: a Swarm Intelligence metaheuristic To find the best restaurant in town each delegate should: 1. Select a restaurant to visit at random (agents restaurant hypothesis). 2. Select meal from the menu at random (partial hypothesis evaluation). 3. IF THEN revisit the restaurant and GOTO (2). 4. ELSE IF the meal a (randomly chosen) friend ate was good THEN adopt their restaurant hypothesis and GOTO (2). 5. ELSE GOTO (1). Unlike an Egon Ronay guide this SDP will naturally adapt to changing restaurant conditions and diner taste over time …

6 Stochastic Diffusion Processes in nature: tandem calling Consider a search for resource (e.g. food) in a dynamically changing natural environment. E.g. Examine the behaviour of the social insects such as: ants (e.g.leptothorax acervorum); honey bees etc. Without a-priori information each ant embarks upon a (random) walk in their environment for a finite period of time. Ants that locate the desired resource return positive. Ants that didnt locate resource return negative. On returning to the nest each positive ant directly communicates with the next negative ant it meets, (non stigmergetic communication). The positive ant communicates the location of the resource by physically steering the negative ant towards it in a tandem pair. Unselected negative ants embark on another random walk around their environment.

7 Compositional Objective Functions In general SDPs can most easily be applied to optimisation problems where the objective function is decomposable into components that can be evaluated independently: … where F i (x) is defined as the i th partial evaluation of F (x). For example SDPs can simply be applied to best-fit string (pattern) matching. Such problems can be cast in terms of optimisation by defining the objective function, F (x), for a hypothesis x about the location of the solution, as the similarity between the target pattern and the corresponding region at x in the search space and finding x such that F (x) is maximised.

8 Partial evidence and inference Assuming a compositional structure of the solution (I.e. objective function decomposable into components that can be evaluated independently) agents perform inference on the basis of partial evidence. Partial evidence for each agents hypothesis [of the best solution] is obtained by a partial evaluation of the agents current hypothesis. Every time a person has dinner at a restaurant the diner selects one meal combination at random from the entire menu of dishes available. Partial hypothesis evaluation allows an agent to quickly form an opinion on the quality of its hypothesis without exhaustive testing. E.g. The Restaurant Game will find the best restaurant in town without delegates exhaustively sampling all the meals available in each.

9 Interaction and diffusion INTERACTION: On the basis of partial knowledge agents communicate their current hypothesis to agents whose own current hypothesis is not supported by recent evidence. E.g. In the Restaurant Game each diner whose last meal was BAD asks a randomly chosen member of the group if their last meal was GOOD: DIFFUSION: If the selected diner enjoyed their last meal then they communicate their current hypothesis, (e.g. the identity of the restaurant they last visited). Conversely, if the selected diner also did not enjoy their last meal then a new restaurant is chosen at random from the entire list of those available.

10 Stochastic Diffusion Processes as global optimisation Central to the power of a SDP is its ability to escape local minima. E.g. Unless all the meals in a restaurant are to a diners taste, then there is a finite non-zero probability that a diners randomly chosen meal will be judged BAD and a new hypothesis adopted. Hence a Stochastic Diffusion Process achieves global optimisation by: probabilistic partial hypothesis evaluation - selecting a meal at random; in combination with dynamic reallocation of resources (agents/diners) via stochastic recruitment mechanisms.

11 Positive feedback mechanisms in SDP In a SDP each agent poses a hypothesis about the possible solution and evaluates it partially. Successful agents repeatedly test their hypothesis and recruit unsuccessful agents to it by direct communication. This creates a positive feedback mechanism ensuring rapid convergence of agents onto promising solutions in the space of all solutions. Hence regions of the solution space labelled by the presence of agent clusters can be interpreted as good candidate solutions.

12 Convergence of SDS AGENT CLUSTERING: a global solution is constructed from the interaction of many simple, locally operating agents, forming the largest cluster. Such a cluster is dynamic in nature, yet stable, analogous to, a forest whose contours do not change but whose individual trees do, (Arthur, 1994). CONVERGENCE: agents posing mutually consistent hypotheses support each other and over time this results in the emergence of a stable agent population identifying the desired solution. E.g. In the Restaurant Game - at equilibrium - a [stochastically] stable group of people with the same hypothesis rapidly clusters around the best restaurant in town.

13 A simple string search Target and search space are defined by the sets of features: T, S. E.g. In a simple string search, component features of the target, T, and search space, S, are alpha-numeric characters. Hypotheses are potential best-fit positions, h, of T in S. The solution is compositional as it is defined by the set of contiguous characters in the search space that together constitute the best instantiation of the target. A population of agents converge on the hypothesis, h, of the best fit position of T in S. Communication between agents is of their hypothesis of the target mapping position, h. Feature evaluation is performed by MATCH (a, b) which identifies if two features (a, b) are similar (defined by a specified similarity metric). Hence the global optimal solution is found at: MAX i MATCH (T[i], S[h+i]).

14 The Stochastic Diffusion Search algorithm INITIALISE (agents); WHILE NOT TERMINATE (agents) DO TEST (agents, T, S); DIFFUSE (agents); END; S is the search space – the text containing the target string T is the target string. Each of the agents maintains a hypothesis, (ie. the best-fit mapping), of the target in S.

15 The INITIALISE phase Assigns each agent a possible hypothesis. I.e. A possible mapping of the target string in the search space text. In the absence of prior knowledge possible hypotheses are generated randomly. A process analogous to: The initial selection of a restaurant at random. An ants initially random walk around their environment.

16 TEST agent activity: randomised partial hypothesis evaluation Partial information on the accuracy of the hypothesis maintained by each agent is obtained by performing a randomised partial hypothesis evaluation. In a simple string search we enquire if one randomly selected letter (i) of the target string, [T i ], is present in the search space, [S], at the position specified by agents current hypothesis, h: i.e. at S [h+i] ? E.g. Is a randomly selected meal good; does a location evaluated by an ant contain resource?

17 An example TEST Target, T: c [a] t Component, (i): search space, SS: T h e c a t s a t Hypothesis, (h) Consider the agent hypothesis to be, (h = 3) Decompose the target, T, & perform partial hypothesis evaluation T i,, (e.g. i = 1) E.g. For (i = 1) the target component symbol, T 1,, is the letter a. Test the agent hypothesis, (h = 3) with partial hypothesis evaluation (i = 1) TEST := MATCH (SS [3+1], T [1]) = MATCH (SS [4], T [1]) = MATCH (a, a) The result of this partial test of the agents hypothesis is POSITIVE As MATCH (a, a) is TRUE.

18 The DIFFUSION phase: stochastic communication DIFFUSION: Stochastically communicates agent mappings across the population of agents. Communication of potentially good restaurants to friends. Communication of potentially good resource locations to other ants. In a passive recruitment SDP each negative agent, (one failing its partial hypothesis test) attempts to communicate with another randomly selected member of the population: Passive recruitment as used in the Restaurant Game. Active recruitment used by leptothorax acervorum. Combined recruitment strategies have also been investigated (Myatt 2006). If the selected agent is positive then its mapping is communicated. Conversely if the selected agent also is negative then a completely new mapping is generated at random.

19 The TERMINATE phase SDS is a global search/optimisation algorithm SDS converges to the global optimal position of the target in search space. A halting criterion examines the activity of the agent population to determine if target has been located. Two such criteria - discussed by Nasuto et al., (1999) - are: The Weak Halting Criterion: Is a function of the total number of positive agents, (I.e. net activity). The Strong Halting Criterion: Is a function of the total number of positive agents with the same hypothesis, (I.e. clustered activity).

20 Algorithm class Global optimisation algorithms have been recently classified in terms of their theoretical foundations into four distinct classes (Neumaier, 2004): incomplete methods: heuristic searches with no safeguards against trapping in a local minimum; asymptotically complete: methods reaching the global optimum with probability one if allowed to run indefinitely long without means to ascertain when the global optimum has been found; complete: methods reaching the global optimum with probability one in infinite time that know after a finite time that an approximate solution has been found within prescribed tolerances; rigorous: methods typically reaching the global solution with certainty and within given tolerances.

21 Algorithm class for heuristic multi-agent systems In heuristic multi-agent systems Neumaier characterisation is related to the concept of the stability of intermediate solutions, because the probability that any single agent will lose the best solution is often greater than zero. This may result in a lack of stability of the found solutions or in the worst case non- convergence of the algorithm. Thus for multi-agent systems it is desirable to characterise the stability of the discovered solutions. For example it is known that many variants of Genetic Algorithms do not converge hence the optimal solution may disappear from the next population. It has been established that the solutions found by SDS are exceptionally stable (De Meyer, Nasuto & Bishop). For example, on SDS convergence in a search with N=1000 agents; search space M=1000; probability of a false negative P - = 0.2 the mean return time to the state of all agents inactive is approximately iterations.

22 Convergence criterion The convergence of SDS was first rigorously analysed by Bishop & Torr (1992) for the case of zero noise and ideal target instantiation. Detailed criteria for SDS convergence under a variety of noise conditions were first discussed by Nasuto et al. (1999), in the context of interacting Markov Chains & Ehrenfest Urn models. However in a recent paper Myatt et al. (2003) outline a much simpler criterion to estimate the suitability of employing SDS for a given search/optimisation problem. Unlike the earlier analysis by Nasuto, Myatts analysis employs two key simplifying assumptions: It utilises the mean number transition of agents between clusters rather than complete probability distributions. It assumes homogeneous background noise. If is the quality of the best solution and an estimate of homogeneous background noise, then the minimum quality required for stable convergence of the algorithm is simply:

23 Time complexity analysis of SDS The Time Complexity of SDS was first analysed in Nasuto et al., (1998) for the case of zero noise and ideal target instantiation. The result has also been demonstrated to hold in the case of convergence under noise. Given M is the search space size and N is the number of agents, one can divide the [NxM] plane into two distinct regions: Region 1 is Linear [ M > M (N) ]: Sequential convergence time is: O (M). Parallel convergence time is O (M/N). Region 2 is independent of search space size M Sequential convergence time is: O (N/log N). Parallel convergence time is: O (1/log N).

24 Conclusions Stochastic Diffusion Procedures constitute a new meta heuristic for efficient global search and optimisation. In a generic search problem - such as string search - the worst case time complexity of SDS compares favourably with the best deterministic one and two dimensional string search algorithms, (or their extensions to tree matching). Further, such performance is achieved without the use of application specific heuristics. Unlike many heuristic search methods (such as Evolutionary techniques; Ant Algorithms; Particle Swarm Optimisers etc.), Stochastic Diffusion Procedures have very thorough mathematical foundations and correspondingly well characterised behaviour.

25 Core references: Ideal convergence of SDS: Bishop, J.M. & Torr, P.H., (1992), The Stochastic Search Network, in Lingard, R., Myers, D. & Nightingale, C., (eds), Neural Networks for Vision Speech & Natural Language, pp , Chapman- Hall. General convergence of SDS: Nasuto, S.J. & Bishop, J.M., (1999), Convergence Analysis of the Stochastic Diffusion Search, Parallel Algorithms, 14:2, pp , UK. Time complexity analysis of SDS: Nasuto, S.J., Bishop, J.M. & Lauria, S., (1998), Time Complexity Analysis of the Stochastic Diffusion Search, Neural Computing 98, Vienna. Simple convergence criteria: Myatt, D., Bishop, J.M. & Nasuto, (2003), Minimum Stable Criteria for Stochastic Diffusion Search, Electronics Letters, 40:2, pp , UK. Change of cognitive metaphor: Nasuto, S.J., Dautenhahn, K. & Bishop, J.M., (1999), Communication as an emergent metaphor for neuronal operation, Lecture Notes in Artificial Intelligence, 1562, pp , Springer-Verlag.

Download ppt "An introduction to search and optimisation using Stochastic Diffusion Processes Stochastic Diffusion Processes define a family of agent based search and."

Similar presentations

Ads by Google