Presentation is loading. Please wait.

Presentation is loading. Please wait.

Game Optimization through Large-Scale Experimentation Erik Andersen Ph.D. student Center for Game Science University of Washington Computer Science Dept.

Similar presentations


Presentation on theme: "Game Optimization through Large-Scale Experimentation Erik Andersen Ph.D. student Center for Game Science University of Washington Computer Science Dept."— Presentation transcript:

1 Game Optimization through Large-Scale Experimentation Erik Andersen Ph.D. student Center for Game Science University of Washington Computer Science Dept.

2 Music Gameplay Sound effects Art

3 engagement retention revenue

4 A/B Testing

5 AB

6 Metrics ● Levels completed ● Time played ● Return rate

7 Aesthetics Secondary Objectives Tutorials

8 Foldit RefractionHello Worlds

9 Refraction

10 Rating: 3.8 / 5 400,000 plays

11 Hello Worlds!

12 Rating: 4.2 / 5 1,385,000 plays

13 Foldit

14 240,000 players

15 Player Tracking ● Flash cache / login name ● New players only

16 Statistical significance 95% confidence level (p<0.05)

17 Experiment #1: Audio Sound Effects Music

18 Result: Music and sound effects did not matter

19 Experiment #2: Animations

20 Result: Animations improved engagement

21 No Animations Refraction

22 No Animations Refraction

23 No Animations Refraction

24 No Animations Hello Worlds

25 No Animations Hello Worlds

26 Experiment #3: Secondary Objectives (Super Mario Bros., Nintendo 1985)

27

28

29 Levels % of players 100 0

30 Levels % of players With Coins Hypothesis

31 Levels % of players With Coins Reality

32

33

34 Time % of players What about time played?

35 Time % of players With Coins Hypothesis

36 Time % of players Hypothesis With Coins

37 Time % of players With Coins Reality: Many Players Quit Sooner

38 Time % of players 100 0

39

40

41 Result: secondary objectives harmed engagement

42 (Assassin’s Creed, Ubisoft 2007)

43 Maybe easier is better? (Super Mario Bros., Nintendo 1985)

44

45

46

47

48 Time % of players Off-path Coins

49 Time % of players On-path Coins

50

51

52 Result: secondary objectives were good if they supported the primary objectives

53 Experiment #4: Tutorials

54 No Animations Foldit

55 No Animations Foldit

56 Refraction and Hello Worlds: no effect

57 Result: text tutorials helped only in the most complex game

58 Number of concepts

59 Context-sensitivity Context-sensitiveContext-insensitive

60 Result: context-sensitive help was better (if tutorials helped at all)

61 No Animations Foldit

62 No Animations Foldit

63 Let’s gather more data ● Let us A/B test your games!

64 Coauthors Yun-En Liu Richard Snider Roy Szeto Seth Cooper Eleanor O’Rourke Jeff Lowdermilk David Truong Zoran Popović

65 Acknowledgements Kongregate: August Brown and Anthony Pecorella Logging: Kefan Xu Refraction Team: Erik Andersen, Yun-En Liu, Marianne Lee, Eric Butler, Brian Britigan, Stephen Sievers, Roy Szeto, Mai Dang, Christian Lee, Ethan Apter, Emma Lynch, Happy Dong, Zorah Lea Fung, Justin Irwen, Seth Cooper, François Boucher-Genesse, Zoran Popović Hello Worlds Team: Rich Snider, Michael Eng, Marianne Lee, Blake Thompson, Jeff Flatten Foldit Team: Seth Cooper, Adrien Treuille, Firas Khatib, Janos Barbaros, Joshua Snyder, Alex Cho Snyder, Jeff Flatten, Jeff Lowdermilk, Dun-Yu Hsiao, Jeehyung Lee, David Salesin, David Baker, Zoran Popović

66


Download ppt "Game Optimization through Large-Scale Experimentation Erik Andersen Ph.D. student Center for Game Science University of Washington Computer Science Dept."

Similar presentations


Ads by Google