1. Dynamic Task Allocation in a turn based strategy game Gilles Schtickzelle September 2012 ULB

2. Problem Statement Creating an intelligent player for a turn-based strategy game. Working Framework: Many possible challenges to meet: o Resource management o Adversarial planning o Spatial reasoning o …

3. A game of FreeCol Colonization of America Establish settlements, grow and develop them Victory : Declare independence & Beat the Royal Expeditionary Force

4. Colony Management Assigning tasks to units for optimal resources production

5. Division of labor in insect societies Ants and wasps colonies have efficient distributed task allocation mechanisms through stygmergy. Bonabeau, E., Theraulaz, G., & Deneubourg, J.-L. (1996).

6. Response Threshold Ants have probabilistic response to stimuli: Varying threshold θ induces specialization o Reduces switching costs o Increases individual efficiency

7. From insects to games FreeCol Colony Units Resources Expert units Ants/Wasps Colony Insects Tasks Specialization

8. Resources Dynamics Surplus: Extra workers. Shortage: Lose worker. Freedom. 50% required to win. Givesbonus or penalty to workers. required to make hammers Used to produce buildings or artileries required to make tools Used to produce buildings or artilleries

9. Allocation Mechanism One stimulus S r for each resource r = One set of dynamic thresholds θ ri per unit i

10. Stimuli and Thresholds Simple computation rules for each stimulus One set of dynamic thresholds θ ru per unit u Genetic Algorithm to find appropriate scale factors β r

11. Simple Scenario

12. AI goals 1.Reach the year 1776 with enough bells to be able to declare independence. 2. Have the best defense possible to resist the attack of the royal expeditionary force. 3.Allocate workers to 1.minimize famine 2.Keep the production modifier as high as possible

13. Results (Basic player) Freedom %SizeFamineMilitary EXPERT100%14024 MEAN (100 games)91.57 ± 2.3915.12 ± 0.850.18 ± 0.0915.99 ± 0.62

14. Planning approach Suboptimal allocation: building too early Two planning methods: o Layered response threshold. o Rule-based planning.

15. Planning approach Layered response threshold : o Use two sets of scale factors: Optimized for growth Optimized for production Rule-based planning :

16. Planning Results (1) Layered AIRule-based AI

17. Planning Results (2) MilitarySoL %SizeFamine EXPERT24100140 MEAN (BASIC)15.99 ± 0.6291.57 ± 2.3915.12 ± 0.850.18 ± 0.09 MEAN (LAYERED)17.49 ± 0.7099.17 ± 0.6514.53 ± 0.840.19 ± 0.10 MEAN (RULE BASED)18.80 ± 0.5498.22 ± 1.2817.49 ± 1.030.16 ± 0.09 Statistics for 100 games with the simple scenario.

18. Modified Threshold rule Unit u produces resource r Unit u does not produces resource r

19. “State of the art” player Modified Threshold update rule + rule-based planning

20. AI players comparison

21. AI goal completion 1.Reach the year 1776 with enough bells to be able to declare independence. 2. Have the best defense possible to resist the attack of the royal expeditionary force. 3.Allocate workers to 1.minimize famine 2.Keep the production modifier has high as possible Freedom %SizeFamineMilitary EXPERT100%14024 MEAN (100 games)100% ± 0.9220.21 ± 0.940.04 ± 0.0720.99 ± 0.55

22. Conclusions Human-level performances can emerge from simple rules, without cheating. Easy to implement (compared to traditional rule-based only AI). Easy to tune down performances (if playing against non- expert). Hybrid system (with planning instructions) improves on basic RTM −Tendency to chaos with large number of stimuli −Difficult to extend to other game aspects (combat, spatial reasoning, diplomacy,…).