1. MENG PROJECT Design of an adaptive robot controller for a predator-prey task using e-puck robots

2. The Goal To design an adaptive robot controller capable of performing a predator-prey task, in a reconfigurable maze

3. Project Specifics Hardware Software Programming in C; Player/Stage; IR Sensors Camera Obstacles Maze

4. Project Breakdown The project can be broken down into 4 separate problems:  Obstacle Avoidance – how to prevent a robot from colliding with obstacles in the maze;  Object Identification – how each robot can identify each other and some additional objects;  Predator-Prey Task – how to make a predator chase a prey and a prey escape from a predator;  Evolution – how to evolve the controllers so that their performance increases. Evolution can also give a controller some adaptability.

5. Behaviour-based Architectures BBTraditional

6. Obstacle Avoidance Infra-red sensors

7. Obstacle Avoidance

8. Rules included

9. Obstacle Avoidance

11. Exploring

12. Monitor the turnrate of the robot for a given period of time; If the variance of the turnrate is below a defined threshold, then the robot must perform a turn; Otherwise, maintain course.

13. Exploring Bigger chance of sweeping the whole maze; Increased number of different perspectives.

14. Exploring

15. Object Identification First solution:  Predator should identify yellow objects as its food;  Prey should identify green objects as its food and yellow objects as predators.  Problem comes up: how to isolate an object of a certain colour in na image?  Image subtraction was the first solution implemented.

16. Object Identification Which images should be subtracted?

17. Object Identification Red ChannelBlue ChannelGreen Channel

18. Object Identification Red - BlueBlue - GreenGreen - Blue Red - GreenBlue - RedGreen - Red  

19. Object Identification Segmenting the chosen images using the Otsu Method  Due to poor results, green is ruled out as a possible colour for the prey’s food. Blue and red are the new candidates.

20. Object Identification Problem found: when no coloured objects are present in the image, the subtraction and Otsu Method give false results.

21. Object Detection New solution: include an mbed board, that has 4 bright blue LEDs.

22. Object Identification Establish new fixed threshold of 205, determined experimentally. New feature: food is available for 3 minutes and unavailable for 20 seconds, cyclically.

23. Object Identification Red ChannelBlue ChannelGreen Channel

24. Object Identification By looking at the images from all 3 channels, one can see that the yellow object is very dark in the blue channel, are bright in both red and green channels. By processing each pixel individually: If both the red and green components of a pixel are 40% larger than the blue component, then that pixel is considered to be yellow.

25. Object Identification Results of the yellow object identification

26. Object Identification How to retrieve information from the thresholded images?

27. Object Identification

28. A scope is then defined for both food searching and for predator avoidance.  Food scope – if the target’s centre of mass is within the scope, no turning occurs. It provides additional stability to the controller.  Avoid scope – if the centre of mass of the object identified is outside of the scope, no turning takes place. If it is within the scope, then the robot must turn to escape. It is the equivalent to the field of view in the natural world.

29. Object Identification

30. Predator-Prey Task The main premises of the task are:  The predator must chase the prey, and get as close to it as possible;  The prey must try to escape from the predator;  The prey, like the predator, also feeds. In its case, the food is the blue light source.

31. Predator-Prey Task An energy level was included in each controller, to reflect the events of feeding and dying. Therefore, new specifications are made:  Each robot starts off with a random energy level between 80 and 100%;  If the robot’s energy falls below a defined Hungry Level, then the robot becomes hungry and starts to look for food. The predator begins searching for the prey, and the prey begins searching for the blue light source;  When a prey is caught by the predator, its energy level becomes 0% (death);  When a predator catches a prey, it eats until it is no longer hungry;  A prey, when eating, should take approximately 3 seconds to eat enough to give it 20 percentual energy points. It should eat until it is no longer hungry;  To be allowed to “eat”, the distance between the robot and the food source must not be larger than 5 cm.

32. Predator-Prey Task Additional features included:  Randomness in rebirth – when a robot dies, it is reborn after a few seconds. When it is reborn, there is a 51% chance it will be reborn with the same role it had before, and a 49% chance it will be reborn with the other role (i.e., predator becomes prey, and vice-versa);  “Cannibalism” – one predator might make the decision of eating another predator. This is possible because of randomness in rebirth;  Role-changing – if a predator who is very low on energy attacks a prey that has a very high energy value, then they change roles;  360 “sweep” – the robot comes to a halt and rotates 360 degrees around itself, hoping to find food.

33. Predator-Prey Task Communication between robots

34. Predator-Prey Task

37. Evolution Birth Life Death EvaluationSelectionMutation

38. Evolution After a robot dies, the following processes take place:  Evaluation – if the controller’s fitness value for the role it is playing is equal or superior to the best fitness value found up to that point, then the controller is selected. Otherwise, the controller gets discarded, and skips to Mutation.  Selection – if the controller is selected, then its parameters become the best for that role, and are stored in the robot’s memory.  Mutation – the best set of parameters for the role it is playing is downloaded from the robot’s memory, and they are slightly changed. Each parameter gets its value changed by [-RANGE; RANGE].  The range is defined for each parameter, and it is percentual. The larger the parameter, the wider the range.  After these tasks are performed, the robot is then reborn with its new set of parameters, and competes until it runs out of energy. The process is then repeated.

39. Evolution

40. How to design the fitness function? First of all, it is important to be aware that if one chooses the fractional configuration for the fitness function: The variables should be included in the function as shown. The three variables chose to compose the fitness function are: o Number of times eaten by the predator; o Number of times that the robot fed; o Ratio between chases in which the robot caught food and total chases that the robot performed.

41. Evolution Keeping the fitness model in mind, the first value has to decrease the fitness, so it goes on the denominator. The other two contribute positively for the robot’s performance, and therefore should go on the numerator of the fraction model.

42. Evolution The parameters chosen to be included in the Evolutionary Algorithm are as follows:  Prey’s Food Gain – gain associated to rule 20, if prey;  Prey’s Avoid Gain – gain associated to rule 21, if prey;  Predator’s Food Gain – gain associated to rule 20, if predator;  Predator’s Food Gain – gain associated to rule 21, if predator;  Size of “food scope” – width of the scope previously mentioned;  Size of “avoid scope” – width of the “field of view”;  Hungry Level – energy level below which the robot becomes hungry;  Energy Danger Level – energy level below which the robot knows it is about to die;  Variance Threshold – the limit that regulates the exploratory behaviour of the robot.

43. Evolution Each robot then has two different sets of parameters: the ones assigned to the predator and the ones assigned to the prey. Since two robots participate in the experiment, and they evolve differently, then 4 sets of paramateres will evolve in different ways.

44. Evolution Standard Configuration refers to when robot A is the predator and robot B the prey. Swapped Configuration refers to when robot B is the predator and robot A the prey.

45. Results The controllers were allowed to run for a few generations, and an interesting result came up: Standard ConfigurationSwapped Configuration

46. Results The prey’s fitness value increased dramatically because a prey-prey scenario occurred. Without anything to decrease its fitness value, the parameters’ path of evolution became corrupted. The environment would make the prey even less suitable to compete with the predator. Therefore, to prevent both prey-prey and predator- predator scenarios, randomness in birth and cannibalism were excluded from the project. Even so, the 4 sets of parameters all get a chance to evolve, since the role-changing still occurs when a predator is weak and catches a strong prey.

47. Evolution Evolution is restarted, and tests are carried out. During the rest of the project, the controllers were left to evolve, without any intereference. The Standard Configuration prey evolved throughout 141 generations, and the predator throughout 29. The Swapped Configuration prey evolved throughout 72 generations, and the predator throughout 19. It is important to be aware that all sets of parameters evolve from the same seed. The original set of parameters was defined through some basic experimentation.

48. Results The fitness values of each generation Standard Configuration

49. Results The fitness values of each generation Swapped Configuration

50. Results To keep it simple, the results will now be focused on the Standard Configuration, that evolved throughout more generations.

51. Results This is how the parameters evolved in Standard Configuration: Prey Predator

52. Results Prey Predator

53. Results

54. However, it is difficult to look at this data and withdraw conclusions. There are, nevertheless, some comments to be made:  The predator develops a much narrower food scope, in order to quickly deal with an evasive prey.  For some reason, the evolution of the Hungry Level and also of the Energy Danger Level parameters in both predator and prey is very similar. What can this mean?  The predator’s variance threshold seems to have a tendency to become smaller, whereas the prey’s seems to be rising. Why? Maybe because the predator already performs periodic 360 sweeps, and therefore does not need to explore the maze as much as the prey.  The avoid scope is the strange result. Since it represents the robot’s “field of view” for the objects it is trying to avoid, shouldn’t the prey develop a wide avoid scope?

55. Results The fitness obviously increased, but has the performance of the controller followed? To prove this, a battery of tests was carried out, creating competitions between the Swapped Configuration predator/prey, the Standard Configuration predator/prey, and also the predator/prey with the original non-evolved set of parameters. For these tests, evolution was halted. Each competition lasted approximately 20 minutes.

56. Results

59. Major observations:  Both predator and prey of the Standard Configuration show the best performances on all tests. The predator shows a fitness improvement of 680%, and when competing against a non-evolved prey it didn’t even die. The prey shows a smaller improvement, of around 66%.  Both Standard and Swapped Configurations appear to have improved the controllers’ performance. The Swapped Configuration, having had less time to evolve, had a milder improvement of performance, of about 115% for the predator, and the prey actually suffered a 3% decrease in performance. This can be due to the fact that the number of generations is simply too small.  Also interesting is the fact that the Standard Configuration prey only dies once without eating when competing against a non-evolved predator. This is a major achievement.

60. Results Major observations:  Also, there are some parameters that evolve in totally different ways in each configuration. For instance, the prey’s avoid gain and avoid scope size have a tendency to become large in the Swapped Configuration, and an opposing tendency to become small in the Standard Configuration. Comparing the performances, maybe it’s better for the prey to have no fear of the predator.  The Standard Configuration predator (the best) has also developed a food scope about 5 times smaller than the Swapped Configuration one. That may prove to be determinant in improving its performance.

61. Conclusions Implementing a behaviour-based robotic controller using only a set of rules proved to be a clean and simple way to solve problems such as Obstacle Avoidance and Exploring, or even chasing a target. The global behaviour of the robot is made up of small individual behaviours, competing and cooperating between themselves. The image processing part was the bottleneck of the project. A lot of time spent on finding alternative solutions to the problem.

62. Conclusions It is also the most sensitive part of the system. If lighting conditions change significantly, the Object Identification might yield false results, compromising the predator-prey task. A larger number of generations would have produced a larger amount of data, which would have been useful to withdraw more conclusions. On one pair of parameter sets, the performance of both predator and prey improved, thus confirming the Red Queen Effect.

63. Conclusions If the maze changes, the parameters should be reset, and evolution should make the robots adapt to new conditions. Without a map of its surroundings, the controllers are unable to come up with new strategies, and they are also unable to calculate either an absolute or relative localization in the maze. Without localization, they cannot effectively understand its surroundings, and therefore adaptation to the environment is not exactly possible, but instead the robots end up adapting to each other.

64. Future Work To add further constraints to the robot’s movement, one needs only to include new rules, and incorporate them in the functions that control the actuators. The program is ready to incorporate a GRN; A camera with improved quality could really boost the performance of the controllers, and allow for simpler processing. A third robot might produce some very interesting developments in the project. With 2-on-1 scenarios, the fitness values might change completely, and even help speed up evolution. Changes to the fitness function could also speed up evolution. For instance, rewarding a prey for the amount of time it stays alive.

65. The End That’s it, thank you for your attention!