1. An Agent that plays Pacman
Abbas Mehrabian, Arian Khosravi
Computer Engineering Department
Sharif University of Technology
{mehrabian,
3 December 2007

2. About the game of Pacman
The program gets its input from the screen!
This decreases speed by ~50%.
2/17/2019
An Agent that plays Pacman

3. An Agent that plays Pacman
Glossary
Pacman
Ghost
Edible
Pill
Power Pill
Snack
Walls and Tunnels
2/17/2019
An Agent that plays Pacman

4. An Agent that plays Pacman
Game objectives
Escape from the ghosts.
Eat all of the pills to finish the level.
When hunting, eat the maximum number of edibles.
Eat the power pills at good times, i.e. when there are lots of ghosts around.
2/17/2019
An Agent that plays Pacman

5. Modeling the game: gametable
Gametable: a 31x31 table that models the game state.
Distance can be defined in the gametable.
2/17/2019
An Agent that plays Pacman

6. Modeling the game: gameobjects
Class GameObject { position in gametable direction distance to other GameObjects }
Pacman, Ghost, Edible, Snack are subclasses of GameObject.
Definition: A connected set is a set of pixels with the same color.
Every connected set may be the graphical representation of some gameobject.
2/17/2019
An Agent that plays Pacman

7. Program outline
Image Processing Phase
while true { capture the screen extract all connected sets for each connected set cs: find a gameObject obj such that cs is the representation of obj update position of obj in gametable
among the four possible positions to go: find the position bestNext such that evaluate (bestNext) is maximum
press the key to go to bestNext }
Note: Since this is an infinite loop, it consumes CPU too much!
Decision Making Phase
2/17/2019
An Agent that plays Pacman

8. Locality of decision making
Agent makes decision only at this moment to go to one of the neighbors.
This looks greedy, why don’t we predict future and devise long-term plans, or use min-max algorithm?
The ghosts are very hard to predict.
The number of possible future game states is large, and it will take a lot of time to calculate deeper. Recall that this is a real-time game!
Greedy is simple to implement, and it worked!
2/17/2019
An Agent that plays Pacman

9. An Agent that plays Pacman
How to evaluate a point?
evaluate (point n) { Compute parameters p1,p2,…,pm for point n return c1p1 + c2p2 + … + cmpm }
Some of the parameters are:
Distance of n to the nearest pill
Distance of n to the nearest ghost
Distance of n to the nearest power pill
The coefficients c1,c2, …, cm are important and should be optimized!
When a coefficient is large, it means that the corresponding parameter is important.
2/17/2019
An Agent that plays Pacman

10. How to meet game objectives?
Escape from the ghosts.
Eat all of the pills to finish the level.
When hunting, eat the maximum number of edibles.
Eat the power pills at good times, i.e. when there are lots of ghosts around.
In order to achieve each of these objectives, some parameters should be defined.
Today I will talk about just the first objective, since it is the most important and challenging!
2/17/2019
An Agent that plays Pacman

11. An Agent that plays Pacman
Escape parameters - 1
The most obvious parameter is distance to the nearest ghost. Try to maximize it.
But, this is not perfect! Look at this figure:
Obviously, it is better if Pacman goes down, but this parameter says that it should go up!
Conclusion: We should find more visionary parameters!
2/17/2019
An Agent that plays Pacman

12. An Agent that plays Pacman
Escape parameters - 2
Definition: Location f is called available from point p, if Pacman can start from p and reach f before any ghost.
Definition: Safeness of a point is the number of available locations from that point.
If a point has more safeness, we are more likely to go there.
2/17/2019
An Agent that plays Pacman

13. An Agent that plays Pacman
Escape parameters - 3
The ghosts rarely reverse their direction, so we can use this property to predict them a little.
Every ghost, spreads its energy throughout the gametable. Most of the energy goes forward.
Definition: Energy of a point is the summation of the ghosts’ energies at that point.
The less energy a point has, the more likely we are to go there.
2/17/2019
An Agent that plays Pacman

14. The relative effect of parameters
The largest a coefficient ci is, the more effect the parameter pi would have on our move.
But we don’t want a certain parameter to have a constant importance in all situations.
For example, escaping parameters should affect more when the situation is red!
Conclusion: The coefficients should not be constant during all phases of the game:
First, we evaluate our current position, and get a feeling about the situation.
Then we set the coefficients.
2/17/2019
An Agent that plays Pacman

15. An Agent that plays Pacman
Implementation
Several situations are defined. In each situation, certain coefficients should be larger:
(before making decision) if a ghost is near or safeness is low then //Situation is “danger” c1 = … … else … endif (now, make decision)
2/17/2019
An Agent that plays Pacman

16. The first version program
Our first version, finished on 14 Sep. 2007, has 4 different situations and 14 parameters!
It gets points on average on my PC.
It has the best official record of an agent that plays this version of Pacman:
2/17/2019
An Agent that plays Pacman

17. Current issues and future work
We have made the game discrete, but this is sometimes very bad:
Devise long-term plans.
There are at least 56 coefficients in the program, which we have tuned our self.
Use learning algorithms to tune the parameters.
In every learning algorithm, there should be some way to evaluate the current playing strategy. Here, the strategy is represented by the coefficients.
2/17/2019
An Agent that plays Pacman

18. An Agent that plays Pacman
Log-based learning
There is another program (called the teacher) that improves the agent’s ability over time.
We assume that there is a human master who plays the game very well.
First, the master plays. At every moment teacher saves the parameters and master’s move in a log file.
Second, teacher reads the log file, and at every moment the agent is provided with the parameters. Agent’s move is logged, and is compared with master’s move.
The more the agent plays like the master, better the strategy is.
Teacher then tries to change the agent’s coefficients to arrive at the optimal strategy.
2/17/2019
An Agent that plays Pacman

19. An Agent that plays Pacman
References
Simon Lucas, “Ms Pac-Man Competition CEC 2007 Results,” University of Essex, Nov. 2007, Nov. 2007,
Any Questions?
2/17/2019
An Agent that plays Pacman