1. An AI Game Project

2. Background Fivel is a unique hybrid of a NxM game and a sliding puzzle. The goals in making this project were: Create an original game experience. Create a challenging AI player using optimized calculations. Achieve an attractive visual look (cute and funny woodland creatures). Accessibility – Easy-to-use UI. The Future? The ability to export the game to the web pon completion.

3. The Game A starting player is chosen randomly.

4. The Game Each turn, a player puts a piece in an empty slot, and slides a tile to an empty place.

5. The Game Each turn, a player puts a piece in an empty slot, and slides a tile to an empty place.

6. The Game Each turn, a player puts a piece in an empty slot, and slides a tile to an empty place.

7. The Game Each turn, a player puts a piece in an empty slot, and slides a tile to an empty place.

8. The Game Each turn, a player puts a piece in an empty slot, and slides a tile to an empty place.

9. The Game The first player to place 5 pieces in a row, column or diagonal at the end of his turn, wins.

10. The Game The first player to place 5 pieces in a row, column or diagonal at the end of his turn, wins.

11. The Game The first player to place 5 pieces in a row, column or diagonal at the end of his turn, wins.

12. The Game The first player to place 5 pieces in a row, column or diagonal at the end of his turn, wins.

13. Development Flash/Flex/AS3 PROS: PROS: Allows to create great presentations and GUI’s easily and fast (vector graphics = win). WWW Accessibility. CONS: CONS: AS3 is considerably slow. Flash is very problematic with deep-recursions (15 seconds limit and render-halting due to its single-threaded nature). Problem: Good presentation VS. Strong and reliable computation.

14. Java PROS: PROS: Fast and reliable for deep recursions - exactly what is needed for minimax search. Everybody knows to develop in Java! CONS: CONS: Harder to achieve the visual look we were looking for. Graphical programming in Java is relatively complicated. Java is not popular for web gaming since the early 2000’s. Development

15. HTML Wrapper JavaScript Interface Flex/AS3 Frontend GUI Java Backend Data Structure, Logic and AI Development Conclusion: Combine them!

16. Data Structure The board is based on simple data structures called “Fivlets”. These are sets of indices that form a winning row, column or diagonal. The board statically stores all the 32 Fivlets in the game. Example for Row Fivlets

17. Data Structure Each Fivlet knows the status of the 5 slots it contains. This structure allows to perform various calculations faster than other methods: Moves are easy to perform (bounded by number of Fivlets the modified slots are in). Example for Column and Diagonal Fivlets Iterations over all Fivlets [O(1)] in order to check for winning conditions and base the heuristics upon their state.

18. AI and Heuristics Automatic players in Fivel use α & β pruning algorithm to search for an optimal move. Fivel has 4 different levels of difficulty: Difficulties are different from each other by a few factors: α & β DEPTH - The search depth for the algorithm. AGGRESSION and DEFENSE RATES - Used in the heuristic scoring. Determine how much importance the player will pay for playing aggressively or defensively. Rascal The Beginner Owlstein The Expert Beer’sa The Tough Fiv-Evil The Godlike

19. AI and Heuristics Fivel’s branching factor is high - Bounded by 128. 32 (slots) x 4 (moveable tiles) = 128 We introduced dynamic depth increment for some of the difficulties: EARLY GAME - The CPU player request a move with a relatively low depth (“quick first move”). After a few turns, the depth increases by 1. MID GAME (mutual turn 10) - The CPU player request a move with a high depth which is now reasonable due to: The decrease in possible moves. Game tree is bounded by 16 mutual turns.

20. α & β Performance Analysis AI and Heuristics Reasonable Time Zone Depth Mutual Turns Time in Secs. (log scale)

21. After analyzing the performance of the algorithm with different depths and aggression/defense rates, difficulties were determined: Search Depth Increase DepthsDefense Rate Aggression Rate Difficulty Level Starts and stays at 1.132 Beginner Starts and stays at 2.222 Expert Starts at 3, increases to 4 in mid-game. 3-422 Tough Starts at 3, increases to 4 quickly and ends at 5 in mid-game. 3-522.5 Godlike AI and Heuristics Fivel’s Difficulty Levels

22. Heuristic Score When a terminal board node is reached (either a board at an end state or when the algorithm reached its designated depth) it is rated according to the following algorithm: For each Fivlet in Board For each Slot in Fivlet If Slot is Mine then mine += 1 Else if Slot is Opponent then opp += 1 If mine == 5 then score = BIG_CONST Else if opp == 5 then score = -BIG_CONST Else if mine == 0 and opp > 0 then score -= -opp^DEF Else if opp == 0 and mine > 0 then score += mine^AGG Return score

23. Basically, the algorithm iterates over all Fivlets and rates each one. Eventually it sums up all the scores and returns it as the board’s heuristic score. For each Fivlet in Board For each Slot in Fivlet If Slot is Mine then mine += 1 Else if Slot is Opponent then opp += 1 If mine == 5 then score = BIG_CONST Else if opp == 5 then score = -BIG_CONST Else if mine == 0 and opp > 0 then score -= -opp^DEF Else if opp == 0 and mine > 0 then score += mine^AGG Return score Heuristic Score

24. The algorithm counts how many pieces the current player and his opponent placed in each Fivlet. Empty (no piece) or Void (no tile) slots are not counted and treated the same. For each Fivlet in Board For each Slot in Fivlet If Slot is Mine then mine += 1 Else if Slot is Opponent then opp += 1 If mine == 5 then score = BIG_CONST Else if opp == 5 then score = -BIG_CONST Else if mine == 0 and opp > 0 then score -= -opp^DEF Else if opp == 0 and mine > 0 then score += mine^AGG Return score Heuristic Score

25. The Fivlet score is now calculated according to various cases: if The current player has a full Fivlet, the Fivlet is rated with a high score. For each Fivlet in Board For each Slot in Fivlet If Slot is Mine then mine += 1 Else if Slot is Opponent then opp += 1 If mine == 5 then score = BIG_CONST Else if opp == 5 then score = -BIG_CONST Else if mine == 0 and opp > 0 then score -= -opp^DEF Else if opp == 0 and mine > 0 then score += mine^AGG Return score Heuristic Score

26. Similarly, if the opponent of the current player has a full Fivlet, the Fivlet is rated with a negative high score. For each Fivlet in Board For each Slot in Fivlet If Slot is Mine then mine += 1 Else if Slot is Opponent then opp += 1 If mine == 5 then score = BIG_CONST Else if opp == 5 then score = -BIG_CONST Else if mine == 0 and opp > 0 then score -= -opp^DEF Else if opp == 0 and mine > 0 then score += mine^AGG Return score Heuristic Score

27. If a player has pieces in a Fivlet without an interference from his opponent, the Fivlet is exponentially scored based on the number of pieces in that Fivlet. For each Fivlet in Board For each Slot in Fivlet If Slot is Mine then mine += 1 Else if Slot is Opponent then opp += 1 If mine == 5 then score = BIG_CONST Else if opp == 5 then score = -BIG_CONST Else if mine == 0 and opp > 0 then score -= -opp^DEF Else if opp == 0 and mine > 0 then score += mine^AGG Return score Heuristic Score

28. Why BIG_CONST and not INFINITY? If more than one Fivlet is full (which is obviously better than one), the board’s score will still be INFINITY (INF + INF = INF). For each Fivlet in Board For each Slot in Fivlet If Slot is Mine then mine += 1 Else if Slot is Opponent then opp += 1 If mine == 5 then score = BIG_CONST Else if opp == 5 then score = -BIG_CONST Else if mine == 0 and opp > 0 then score -= -opp^DEF Else if opp == 0 and mine > 0 then score += mine^AGG Return score Heuristic Score

29. AGG and DEF? What has better priority? Playing aggressively (striving for full Fivlets) or defensively (preventing opponent’s full Fivlets)? For each Fivlet in Board For each Slot in Fivlet If Slot is Mine then mine += 1 Else if Slot is Opponent then opp += 1 If mine == 5 then score = BIG_CONST Else if opp == 5 then score = -BIG_CONST Else if mine == 0 and opp > 0 then score -= -opp^DEF Else if opp == 0 and mine > 0 then score += mine^AGG Return score Heuristic Score

30. AI Analysis Average Game Length (Mutual Turns) ResultsNumber of Games Tested Match Type 13Victories 40% Draw 60% 25Human VS. Human 11Human won 15% CPU won 65% Draw 20% 25Human VS. CPU (Beginner) 10,Human won 12% CPU won 73% Draw 15% 20Human VS. CPU (Expert) 13Human won 0% CPU won 85% Draw 15% 15Human VS. CPU (Tough) 11Human won 0% CPU won 90% Draw 10% 15Human VS. CPU (Godlike) Human

31. AI Analysis Average Game Length (Mutual Turns) ResultsNumber of Games Tested* Match Type 9Victories 90% Draw 10% 10CPU (Beginner) VS. CPU (Beginner) 15Weaker CPU won 0% Stronger CPU won 10% Draw 90% 10CPU (Beginner) VS. CPU (Expert) 8Weaker CPU won 40% Stronger CPU won 50% Draw 10% 10CPU (Beginner) VS. CPU (Tough) 14Weaker CPU won 10% Stronger CPU won 40% Draw 50% 10CPU (Beginner) VS. CPU (Godlike) Beginner Difficulty *More than one game tested since there’s a small random factor in CPU vs. CPU games (generated moves and starting player).

32. AI Analysis Average Game Length (Mutual Turns) ResultsNumber of Games Tested* Match Type 14Victories 30% Draw 70% 10CPU (Expert) VS. CPU (Expert) 10Weaker CPU won 20% Stronger CPU won 70% Draw 10% 10CPU (Expert) VS. CPU (Tough) 14Weaker CPU won 30% Stronger CPU won 10% Draw 70% 10CPU (Expert) VS. CPU (Godlike) Expert Difficulty *More than one game tested since there’s a small random factor in CPU vs. CPU games (generated moves and starting player).

33. AI Analysis Average Game Length (Mutual Turns) ResultsNumber of Games Tested* Match Type 7Victories 90% Draw 10% 10CPU (Tough) VS. CPU (Tough) 9Weaker CPU won 50% Stronger CPU won 30% Draw 20% 10CPU (Tough) VS. CPU (Godlike) Tough Difficulty *More than one game tested since there’s a small random factor in CPU vs. CPU games (generated moves and starting player).

34. AI Analysis Average Game Length (Mutual Turns) ResultsNumber of Games Tested* Match Type 15Victories 30% Draw 70% 10CPU (Godlike) VS. CPU (Godlike) Godlike Difficulty *More than one game tested since there’s a small random factor in CPU vs. CPU games (generated moves and starting player).

35. AI Analysis Conclusions Why so many Draws? Why so many Draws? Fivel’s board becomes saturated roughly after mutual turn 11 if both opponents manage to block the Fivlets of each other. On average, if a game doesn’t end after mutual turn 11, it ends in a full-board draw, since it’s very hard to find un- blocked Fivlets. An exception to this is the Godlike VS. Godlike match. While 70% of the games ended in a draw, the ones that ended in victory for one of the players, did so in the last playable turn.

36. AI Analysis Conclusions Why are there anomalies in CPU vs. CPU matches? Why are there anomalies in CPU vs. CPU matches? In some matches the weaker difficulty player has more victories than the stronger difficulty (or most games ended in a draw). We do not have a definite answer, but there might be a few explanations: Heuristic scoring is strong enough even in low depths of search. Starting player (random factor) has lots of significance. Board saturation.

37. AI Analysis Conclusions Is the heuristic scoring really that strong? Is the heuristic scoring really that strong? We tested many BEGINNER vs. HUMAN matches and found out that most players had trouble beating that difficulty (at least in their first Fivel games). Nevertheless, we discovered that the starting player had an advantage in this difficulty level. This didn’t happen in harder difficulties.

38. Enjoy the Game! woot?