The Effects of Loss and Latency on User Performance in Unreal Tournament 2003 Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, Mark Claypool

Introduction More and more people play networked computer games Understanding of network issues as they relate to games will help game designers, network engineers, etc. Lots of studies done for racing games, RTS, “custom games”, but not so much for FPS games. Specifically, want to study packet loss and its effects on user performance.

Introduction, cont. Gathered latency, loss numbers from live UT2k3 servers as a base point. 4 fundamental FPS interaction components:  Movement  Precision Shooting  General Shooting (Spamming)  Moving and shooting simultaneously Designed experiments with maps to isolate each component.

Background Focused on 2 game-play types: Deathmatch and Capture the Flag Weapons: Subjectively divided into 4 categories, based on “Precision Required”  High: Shock Rifle, Link Gun, Lightning Gun  Medium: Assault Rifle, Mini-gun, Bio Rifle  Low: Flak Cannon, Rocket Launcher, Redeemer, Ion Painter  Other: Shield Gun, Translocator, Ball Launcher Problems?

Approach Categorize user interactions Construct test environment Conduct pilot studies to determine ranges for typical loss and latency values. Conduct numerous user studies on custom maps using the range for loss/latency values. Analyze!

Approach, cont. Hypothesis? Movement test:  Running pre-defined routes in regular game maps  Simple test: Run a straight line!  Complex test: run, jump, spin, and pickup items. Shooting tests:  1 player with lightning gun trying to hit another player who is just dodging  1 player vs. bot with “medium” precision guns on insta-gib.  Full array during full game to study movement + shooting.

Approach, cont. – Pilot Studies 80% of all servers have no measurable loss Highest measured loss just over 3% 40% have latencies less than 100ms 40% have latencies between 100ms and 140ms 20% have latencies greater than 140ms Established ranges:  Loss: 0% - 6%  Latency: 0ms – 400ms

Approach, cont. – User Studies During a 1 month period, ran over 200 experiments representing hours of FPS game play. All experiments run on testbed. All users “very familiar” with UT2k3 Process:  Dry run with no latency  Operator introduced random latency/loss, users “blind” to network status.  Archived data for later analysis.

Analysis - Movement Straight line “race”. Latency/loss had no effect. Why? Complex course: No effect up to 300ms, slight upward trend beyond that.

Analysis – Precision Shooting Loss alone has no impact. Latency has large effect when greater than 75ms

Analysis – Restricted Game DM Again, loss had no impact Latency had impact on both kills and deaths.

Analysis – Full Game DM Once again, loss has no impact Latency had only a small impact, why?

Analysis - Network Constant bitrates across latency/loss rates. Clients send updates every 10-20ms Server sends updates every 50ms.

Analysis - Subjective Players noticed latencies of 75ms. “Less Enjoyable” over 100ms. Loss noticeable at 3% Not really noticed during movement tests Highly noticeable during precision shooting.

Conclusion Packet loss has no impact on performance, rarely noticed up to 5%. Latency has no impact on movement Latency has great impact on shooting precision, especially over 75ms. Players tend to adapt strategies to combat high latency