1. Colyseus: A Distributed Architecture for Online Multiplayer Games
Ashwin Bharambe, Jeffrey Pang, and Srinivasan Seshan
Carnegie Mellon University

2. Online First-Person Shooters
Multiplayer architecture for multiplayer first-person shoot games.
low latency required
weakly consistent state tolerable
spatial contiguity allows pre-fetching
Traditionally this is done with a client-server model
leads to bottlenecks
can only sustain user numbers in the dozens

3. Colyseus Single-copy replication model
maintains consistency by serializing updates
mirrors existing server model
cuts latency (reading from local copy), but sacrifices consistency in game-wide state
DHT lookup for object queries
both random and range-query DHTs implemented
range-query DHT works well since object queries will always be in contiguous spatial regions
DHT query delay overcome by anticipating which objects will be needed soon and pre-loading them

4. Replica Manager
Follows Tunable Availability and Consistency Tradeoffs (TACT) model
depending on specific game characteristics, developers can select either more availability or consistency - consistency lowers availability (increases lag), availability lowers conistency.
synchronizes replicas to primary
changes are delta-encoded, sent to primary, then distributed serially to all replicas
when a node becomes interested in a replica (i.e. the player is near that object), it registers with the primary and receives updates directly (decoupled discovery/synchronization)
creates new replicas
deletes replicas that are no longer needed
fast moving objects (missiles) use a special case attachment so that they are automatically sent to nodes that request the object they are attached to (the person who fired the missile)

5. Range-Queriable DHT adjacent nodes responsible for adjacent keys
both standard random DHT and rangeDHT implemented with Mercury
adjacent nodes responsible for adjacent keys
player x,y coordinates used as key, then game can request other objects that are near the player from adjacent nodes
predictions based on current player motion can be used to pre-load upcoming objects
with a known average DHT lookup time, the prediction can be tweaked so that objects finish loading about when they are needed

6. DHT comparison

7. Evaluation Experimental Setup Modified Quake II to work with Colyseus
Emulab used to simulate virtual servers
no link capacity constraint, but ties end-to-end latency to measured samples
artificially dilate time to counter slow virtual servers
model game based on density statistics for a quake III map ( Zipf distribution)
Modified Quake II to work with Colyseus
Mercury rangeDHT
variable size bounding box corresponding to visible objects for a character used as area-of-interest
client/server messaging remains intact so unmodified engines could connect to any of the p2p nodes as if it were a server
Custom map w/ bots for workload
Emulab testbed

8. Results

9. Discussion
Colyseus enables multiplayer first-person-shooter games to handle hundreds of players, instead of dozens
since FPS games have very high demands for low latency and consistency, extending this architecture to other game types, like role playing games, is very feasible
adaptation of commercial Quake II shows that this is feasible for other games in a production environment
this method opens up many possibilities for cheating (a node could be modified to request objects that it shouldn’t ‘see’, for instance), but more work could be done to address those threats.

10. Related Work Real-time strategy (command & conquer)
parallel simulation often used, since consistency is very important
often limited to less than 10 players
Online role playing (world of warcraft, second-life)
cell based
centralized server or server cluster
Distributed Virtual Reality Environments
similar goals to Colyseus, but specific
not catering to common game applications

11. Issues
why not just evaluate actual gameplay? (custom map, all bots seems a little suspect)
synchronization decoupling seems to introduce a bottleneck. It’s still better than a server model since a primary replica might be the only one on a node, but the node still has to handle all traffic for that replica
how are node failures (player logs off) handled?
how do you handle updates on objects in a view that is already inconsistent, especially since the node cannot know if it’s view is consistent.