1. By Dr Simon Martin CHORDS Group Division of Computing Science and Mathematics School of Natural Sciences University of Stirling, Stirling FK9 4LA. spm@cs.stir.ac.uk A Multi-agent based Cooperative Approach to Scheduling and Routing

2. Contents Introduction – What are multi or intelligent agents? Multi/Intelligent -agents IEEE FIPA agent standard MACS Agent-based platform Case Studies VRP PFSP Fairness Fairness in nurse rostering Fairness in requirements assignments for the next release problem Conclusions Future Work Thank you

3. Multi/Intelligent -agents Agents maintain an internal representation of their environment. They communicate by Asynchronous messaging. They are autonomous, no one process is in overall control They are capable of completing a task on their own or can cooperate This means they can execute distributed algorithms where no agent is in overall control

4. IEEE FIPA standard There is an IEEE standard called the Foundation for Intelligent Physical Agents (FIPA) There are are number of Open source FIPA platforms: FIPA-OS http://fipa-os.sourceforge.net/index.htm Jadex Agents http://www.activecomponents.org/bin/view/About/New+Ho me JIAC Intellient Agents http://www.jiac.de/ JADE http://en.wikipedia.org/wiki/Java_Agent_Development_Fr amework

5. FIPA compliant Multi-Agent Platform AMS DF AMS DF The DF (Directory Facilitator) provides a directory which announces which agents are available on the platform. The AMS (Agent Management System) controls the platform. Is the only one who can create and destroy other agents, destroy containers and stop the platform. Inter platform communication

6. Multi-Agent Cooperative System(MACS) Meta-heuristics require careful tuning to a specific problem They require parameter tuning Balancing intensification and diversification Some meta-heuristics are better at some problems than others They have different strengths and weaknesses But what if there was a of combining these strengths and weaknesses in one system? This might be achieved if different meta-heuristics cooperated with each other

7. MACS Problem definition Launcher Agent Cooperating Agent The Launcher Agent (LA) sends the same problem to each agent

8. MACS Launcher Agent Cooperating Agent Agents cooperate by passing Best edges

9. MACS -again.... Problem definition Launcher Agent Cooperating Agent Each agent sends its best overall solution to the launcher agent. The LA takes the best And writes it to file

10. MACS – just to ram it home

11. Multi/Intelligent - agents Image: Wikipedia by Utkarshraj Atmaram. http://en.wikipedia.org/wiki/Intelligent_agent#mediaviewer/File:IntelligentAgent-Learning.png

12. Inside a Multi/Intelligent -agent

13. Ontology for Scheduling and Routing Graph Edge Constraints Vertices CitiesJobsAssignments The Vertex object Is the interface between the framework and specific Problem instance Problem specific data interface Objects of the agent-based framework Problem specific objects inheriting from the abstract vertex object Subgraph Customers & Depots

14. Cooperation protocol

15. Case Studies Permutation Flowshop Scheduling Meta-heuristic Randomised NEH A Juan et. al Capacitated Vehicle Routing Randomised Clarke Wright Savings Algorithm A Juan et. al Fairness In Nurse Rostering VNS, Simulated Annealing and Tabu Search Martin, Smet, Ouelhadj, Vanden Berghe, Özcan. The platform has been applied to three case studies

16. Permutation Flowshop Scheduling

17. Taillard benckmark instance tai_051_50_20

18. Capacitated Vehicle Routing

19. Augerat Benchmark instance A-n63-k9

20. The Nurse Rostering problem The Scheduling of hospital personnel is Particularly challenging because: There are different staffing needs on different days and shifts Staff work in shifts Healthcare institutions work around the clock The need for day and night shifts The correct staff mix for each ward Many different employment contracts Part-time Special arrangements Fairness so that staff are happy

21. The standard objective function Let C be the set of constraints. W c is weight associated with a given constraint N is the number of violations of that constraint. Is the number of roster constraints MinWS = minimise the sum of the sum of all nurses violations Models of Fairness

22. New Fairness objective functions MinMax = minimise the number of nurses × worst nurse violation MinDev = minimise the sum of deviations from the average + the numbers of nurses × the mean roster quality

23. MinError = minimise the sum of the differences of max roster value – min roster value a + the mean roster quality MinSS = minimise the sum of the squared violations associated with assigning a nurse to a given roster Models of Fairness

24. Measuring fairness is done with the Jains Fairness function (Jain et al., 1984; Muhlenthaler and Wanka, 2012). It is the sum of the squared violations in assigning a nurse to a given roster divided by the number of nurses times the squared value of assigning a nurse to a roster. Its values range from the worse case 1/N to 1 where N is the number of nurses to 1 where the roster is completely fair.

25. Fairness Results

27. Tensor Online learning The agent system has been updated: A new learning system has been developed based on tensors It still uses the same conversation structure as before Instead of sharing edges the agents now share tensors made from incumbent solutions.

28. Cooperation Protocol with Tensors

29. Tensor Online learning Agents are 20 best incumbent solutions. The initiator agent, for that conversation, collects all the incumbent solutions. The initiator then builds an tensor where n is the length of problem instance and m is the number of incumbent solutions. The tensor build from adjacent matrices of each incumbent solution. The initiator factorises the matrix. The result is an matrix called a basic frame. The basic frame is treated as an adjacent matrix and converted back to a list of good edges. This list is shared with all the agents. The agents update their short-term memories. The agents then use the list of edges in short-term memory in conjunction with their metaheuristic to build new incumbent solutions.

30. Conclusions Distributed asynchronous agent platform Modular Ontologies Peer to Peer Scalable

31. Future Work Fairness in requirements assignments for the next release problem Model each customers requirements on an agent Compare multi-objective approach to single objective approach Improve the ontology to work on more problems Improve the tensor learning system

32. Papers Simon Martin, Djamila Ouelhadj, Pieter Smet, Greet Vanden Berghe, and Ender Ozcan. Cooperative search for fair nurse rosters. Expert Systems with Applications, 40(16):6674-6683, 2013. Simon Martin, Djamila Ouelhadj, Patrick Beullens,Ender Ozcan,Angel A. Juan,Edmund.K.Burke. A MULTI-AGENT BASED COOPERATIVE APPROACH TO SCHEDULING AND ROUTING. under review European Journal of Operational Research October 2015. Shahriar Asta, Simon Martin, Ender Ozcan, Edmund Burke. A Multi-agent System Embedding Online Tensor Learning for flow shop Scheduling. Submitted to Information Sciences, July 2015.

33. Thank you