1. Game Analytics

2. Interactive Digital Entertainment
User
Digital/ Social Media
Behavioral economics
Game Industry
HCI
Informa-tion Science
Computer Science
Digital storytelling, online behavior
Persuasion, value, learning
User behavior,
data mining
Development, game economics
Communication in games
Play experience, design

3. Personal background MSc. In Natural Sciences PhD in Computer Science
Large-scale trends and evolutions in time/space, Geographic Information Systems
PhD in Computer Science
Empirical evaluation of games, HCI, user testing, game telemetry
Post doc. At the Center for Computer Games Research, IT University Copenhagen
Play experience, biometrics, game data mining, game development
RA/project lead, Department of Informatics, Copenhagen Business School
Game piracy, behavioral economics, co-creation – more game telemetry data mining
Assistant Prof., Department of Communication, Aalborg University
Yet more game data mining, more game development, more innovation
Co-Founder & Lead Game Analyst, GameAnalytics
Tools and consulting on application of game telemetry to development

5. Research breakdown 90% applied research
10% theory (play experience, play personas)
Collaboration with industry – real needs
Collaboration with international colleagues
1 single-authored publication ...

6. Game user research

7. Game User Research
Focus: How users interact with IDE applications and each other + the business side
Game User Research – answering e.g.:
Who are the users of interactive digital entertainment products?
What do they do and where, with whom and why?
How do we develop products for different users?
GUR is a nebulous concept at best, reflecting that user-oriented research in IDE is relatively young

8. Game User Research Multi-disciplinary ”field”
Researchers from CS, HCI, communication, design, media, psychology, AI, art, economics, development ...
Emergent field – lack of established theory
Exponential growth in research publications
Backed by a growing industry where users are central

9. Game User Research Four main lines of investigation in GUR:
Usability: Can the user operate the controls?
Playability: Is the user having a good experience?
Behavior: What is the user doing while playing?
Development: Integrating GUR in business practices

10. Game User Research Why interesting? New field Multi-disciplinary
New field of research
Emerging methodologies + theories
Plenty of tough problems
Collaboration
Broad relevance
Multi-disciplinary
Affects millions of people
Industry interest
Latest technologies
New field
Multi-disciplinary
Impact

11. Interactive Digital Entertainment
User
Digital/ Social Media
Behavioral economics
Game industry
HCI
Informa-tion Science
Computer Science
Digital storytelling, online behavior
Persuasion, value, learning
User behavior,
data mining
Development, game economics
Communication in games
Play experience, design

12. Patterns in play behavior Behavior correlations with PX/PsyPys/design
User behavior
Patterns in play behavior
Play personas
Spatial user behavior
Behavior correlations with PX/PsyPys/design
Understanding games and players Improving development & testing

13. Game metrics

14. What are game metrics? Metrics = Business Intelligence [BI]
BI is derived from computer-based methods for identifying, extracting and analyzing business data
for strategic or operational purposes
Across market-, geographic- and temporal distance
Supports decision making (Decision Support Systems)

15. What are game metrics? Quantitative measures about any aspect of games
Players: gameplay, customers, monetization,
Production: team size, pipeline, milestones, markets
Technical performance: servers, infrastructure
Any other relevant quantitative measure (e.g. management)
Analysis of game metrics = game analytics
[No accepted definition (working on a standard)]

16. What are game metrics? Metrics are measures, e.g.:
Average playtime per player
Number of ”Swords of Mayhem +5” sold
Daily Active Users
% server uptime/stability
Avg. network latency
Bugs reported/bugs resolved /day
Customer support call avg. length
Players
Performance
Process

17. Players

18. Players

19. Performance

20. Process

21. Process

22. Why game metrics analysis?
Big data: populations not samples
Understanding all players
Research/development out of the lab and into the real world
Big depth: Detailed recording of all aspects of play
Includes communication, navigation, cross-games ...
Combining GUR data sources for in-depth research

23. Challenges
Behavioral telemetry inform what players are doing, only by inference why
Finding the right features to track is not obvious
Managing the allure of numbers

24. Game Data Mining

25. Game data mining Game data mining = data mining of game metrics
Gartner Group:
“the process of discovering meaningful new correlations, patterns and trends by sifting through large amounts of data stored in repositories, using pattern recognition technologies as well as statistical and mathematical techniques”

26. Game Data Mining: approaches
Common approaches in game data mining:
Description
Characterization
Discrimination
Classification
Estimation
Prediction
Clustering
Association

27. Description Simple description of patterns in data
Accomplished using Explorative Data Analysis
Example: how rapidly does the ”warrior” class advance through levels?
Answers many questions from designers and producers

29. Description
Drill down/across

30. Prediction
Using a large number of known values to predict possible future values
How many players will an MMORPG have in 3 months?
When will a F2P break the 1 million player threshold?
When will people stop playing?
One of the most widely used data mining methods in game analytics
Persistent world games
MMOs
F2P

31. Clustering
Orders data into classes, but the class labels are unknown (unsupervised)
Groups formed according to internal similarity vs. across-group dissimilarity
Subjective element
Problems applying
algorithms to game metrics

32. Player Behavior in Tomb Raider: Underworld w/ Alessandro Canossa, Georgios Yannakakis, Julian Togelius, Hector Perez, Tobias Mahlman

33. Behavior in TRU
Goal: Using gameplay (behavior) metrics to classify the behavior of users
Uses:
Comparing behavior with design intent
Optimization of game design
Debugging of playing experience
Adaptation: Real-time dynamic adaptation to player type

34. Behavior in TRU Tomb Raider: Underworld (2008)
AAA-level commercial title
Data from 1.5 million users via Square Enix
Hundreds of variables
Metrics should fit purpose
Selected variables fitting key game mechanics
Jumping, completion time, causes of death …

35. Behavior in TRU Analysis: Revealed a 4 distinct behaviors (94% users)
Clustering algorithms (PCA, k-means)
Self-Organizing Map (unsupervised)
Revealed a 4 distinct behaviors (94% users)
Players use the entire design space
Behaviors translated into design terms

36. Behavior in TRU
8.68% (Veterans): Very few death events (environment). Fast completion times. Generally perform very well in the game.
22.12% (Solvers): Die rarely, very rarely use the help system. Slow completion. Slow pace of play.
46.18% (Pacifists): Largest group of players, dies from enemies. Fast completion time, minimal help requests. Good navigation skills, not experienced with FPS-elements in TRU.
16.56% (Runners): Die often (enemies, environment), uses the help system, very fast completion time

37. Behavior in TRU Towards big data: 1st study: 1365 players
2nd study: 30,000 players
3rd study: 203,000 players
4th study (in prep): 1.6 million players
5th study (in prep): across games
From dozens to hundreds of variables

38. Behavior in TRU Can we predict when people stop playing?
Use: uncovering design problems; engagement
Approach
TRU: 7 levels + prologue
10,000 randomly selected players
7 groups of metrics (400+ variables)
Training data: lvl 1
Simple logistic regression best fit: 77.3% (base: 39)

39. Behavior in TRU Decision trees (prediction)
Use: predicting player behavior; transparent models – ideal for communicating across stakeholders
Level-2 rewards
Rewards > 10
Level-3 playtime
-> playtime > 43 minutes : 4
-> playtime < 43 minutes : 7
Rewards < 10 : 2
Lvl 2 rewards and playtime lvl 3 predictors of quitting

40. Fun Facts about Character Names w/ Christian Thurau

41. Character names Do paladins always have names like ”Healbot”?
Do Warlocks always have names like ”Ûberslayer?”
Are mages always called ”Gandalf”?
Are there any kind of ?

42. Character names 7,938,335 WOW characters (5 years logging)
Name, Race, Class, Playtime, Guild, Server Type, Domain, etc. ...

43. 3,803,819 Some findings unique names (a surprising lot)
More diverse than real-world names - despite naming constrictions
Looks like naming is important to players – only unique feature you have
RP-characters most diverse (83% unique – rest ~58%)

44. Any relationships between name and game features?
Class

45. Any relationships between name and game features?
Race

46. Races
”Pretty”
”Bestial”

47. 2D-isomap projection (dimensionality reduction technique)
”pretty” races named differently than ”bestial” races
Not due to differences in m/f character ratios

48. Gnomes and dwarfs named as ”bestial” races?=
Gnomes and dwarfs named as ”bestial” races? =

49. RP vs. PvP/PvE servers Names on US servers different from EU servers
Except for RP realms (larger overlap btw. EU/US)

50. Can we predict names? What is the chance that ”Gimli” will be a dwarf?
Estimated conditional propabilities of a given class/race/server type given a particular character name
Class and Race best predictors, but server type and faction also hints at naming decisions
Some names are very good predictors, others are not -> so yes, Gimli will likely be a dwarf

51. Sources of inspiration

52. Sources of inspiration
1000 most common names
128,058
(not a lot, but still 100* bigger than any other study)
38 coding categories found
Some names multiple categories/hard to classify (e.g. ”Raziel”)

53. Sources of inspiration
Regular vanilla real-world names most common (Sara, Mia, etc.) [186]
Mythology – notably Greek [164]
Anubis, Odin, Ares, Loki, Nemesis
Popular culture – games, cartoons, film ... [174]
Naruto, Sakura, Tidus, Valeria, Revan, Zelda
Fantasy literature (Tolkien rules supreme) [39]
Earendil, Sonea, Morgoth, Aragorn
A lot of names in breach of ToU

54. Sources of inspiration
697 of 1000 names categorized
Rest: Nouns, verbs of unspecified nature
Semantic nature to categorize:
”Negative”: Nightmare, Sin, Fear, Requiem
”Positive”: Hope, Love, Pure
”Neutral”: Who, Moonlight, Magic, Snow

55. Sources of inspiration
Names with negative semantic meaning 6 times more common than positive semantic
Are gamers depressed?
Or do ”dark” names just sound cooler?

56. Perspectives Lots of ”why”´s unanswered:
Why are certain names more popular than others?
Why do the Mage class exhibit a greater variety of names than other classes?
Why do some players pick names of characters from the same game they are playing?
Need to talk to the players ...

57. Future

58. Future Demand in the IDE industry Attractive research challenges
Unique openness to research-industry collaboration
Attractive research challenges
Complex, mixed-methods, multi-disciplinary, big data

59. Future Better methods and algorithms (all forms of metrics)
Correlating behavior, PX and design
Spatial game analytics
User profiling: behavior, personality, motivations ...
Decoding and predicting behavior
Maturing development practices (from joint warehousing to GUR)
”Guerrilla metrics”-methods

60. Playtime and power laws w/Fraunhofer IAIS
6+ games
5+ game ”types”
Same patterns?
90%+ prediction
Power law:
When the frequency of an event varies as a power of some attribute of that event (session length)
Does all playtime behavior follow a power law?

61. Thank you

62. Want to know more? Blog.gameanalytics.com andersdrachen.wordpress.com
[slide deck available here]
IGDA GUR SIG – LinkedIn group, 350+ members
The GUR SIG Mendeley Library – mixed industry/research
GDC archives – industry SOTA
Research publications – ACM, IEEE, Springer digital libraries + new book on game telemetry out 2012