1. Games User Research Cyril Rebetez User Research Team Sony Computer Entertainment Europe { 1

2. User Research at Sony PlayStation 2 1. Who are Sony Computer Entertainment? 2. Games user research vs usability testing 3. How to do games user research Recruitment Methods Analysis Reporting (Break) Activities – setup a games user research

3. Who are Sony Computer Entertainment? 3

4. 4

5. What does the User Research Team do? … 5 Usability and User Research – During Development

6. Now is a good time to watch a video 6

7. What does the User Research Team do? 7

8. Research Projects What does the User Research Team do? 8

9. 9

10. Games user research vs Usability? 10

11. usability effectivenessefficiency memorability learnability satisfaction Productivity

12. G.U.R effectivenessefficiencyEngagement

13. Not designed for efficiency 13

14. Games must challenge 14 (in the right way)

15. Passionate Audience 15

16. Different interaction methods 16

17. Different interaction methods 17

18. Different interaction methods 18

19. G.U.R  Reduce Hassle  Maximise productivity  Consistent simplicity  Understanding  Focus on effectiveness & efficiency  Mainly behaviour & some attitude  Reduce interface Hassle  Maximise engagement  Progressive complexity  Understanding & performing  Focus on fun & emotion  Both behaviour & attitude usability

20. 20 Researching User Experience

21. How to do it? 21

22. 1. Kick off – What are we looking for? 2. Preparation – How do we do it? 3. Recruitment – Who should do it? 4. Moderation – How to make them do it? 5. Observation – What to look for? 6. Analysis – What does it mean? 7. Reporting – How to explain it? How to do it? 22

23. 1. Kick off 23

24. Objectives 24 User Research kick-off Meet with the people who make the game, What do they want to know? Everything Define the objectives Who are the audience? Decide how we can answer their questions.

25. 2. Methods 25

26. 1 to 1 testing 26

27. 1 to 1 testing 27 Deep testing with 1 user at a time Focus on observation and interviews

28. 1 to 1 testing What questions can it answer? understanding Anything based around understanding, including: Do players know what to do? Do players know where to go? Do players know how to use this weapon or ability? What do players think about a feature? … 28

29. 1 to 1 testing 29 “Make sure the VITA can see the cards”

30. 1 to 1 testing 30 “Make sure the VITA can see the cards”

31. 1 to 1 testing 31

32. 1 to 1 testing Why is it suitable? Allows close observation of the issues Can ask questions to the players 32

33. Multi-seat testing 33

34. Multi-seat testing 34

35. Multi-seat testing 10 users at a time Focus on quantitative data 35

36. Multi-seat testing What questions can it answer? Are there any difficulty spikes? Can players complete the game? How long does it take players to learn all of the mechanics? And more… 36

37. RESOGUN 37

38. Multi-seat testing 38

39. Multi-seat testing Why is it suitable? Many participants increase reliability Questionnaires can get quantitative data Can test longer sections of gameplay 39

40. Home Testing Real world setting Uncover issues you can’t find in a lab 40

41. Home Testing 41

42. Home Testing 42 ?

43. Home Testing 43 ?

44. Home Testing 44

45. Diary Studies Real world setting Uncover issues that reveal themselves over time 45

46. Diary Studies 46

47. Diary Studies 47

48. Telemetry Interstitial questionnaires RITE Usability Evaluation And more… 48

49. 3. Recruitment 49

50. Audience

51. Diversity

52.  More than 34% of gamers are 18+ females  Less than 18% of gamers are under 17 males  25% of the gamers are 50+ years old  Average age of gamers is 35 years old Diversity

53. o Use market research outcomes o Define with stakeholders o What is the vision? o Who is likely to play / purchase / use the product? o Is it an established franchise? o Trying to break into a new market? Defining users profiles 53

54. How many users? 54 “5 users will typically uncover 80% of the usability issues” that the whole target audience will experience (Nielsen, 1998) Usability :

55. Recruitment Group profile definition  Demographics (age, gender)  Gaming profile: Hardcore, Core, Casual, Fanatic, Kids, etc…)  Type(s) of games played  Hours of play per week (online & offline)  Number of games purchased in the last 3 months  Games & system(s) owned (e.g. 360° + Kinect)  Other criteria (e.g. franchise awareness, play preferences, etc.)

56. Recruitment Example of screener criterionrequirementscreener Age range(s)5x teens 13 to 16 (equal spread)Discard: 16 yrs old Gender(s)3x boys & 2x girls Gaming habit2 to 5 hrs/week offline & 0 onlineDiscard 5 hrs/week purchaseMax 2 game in last 3 monthsDiscard >2 games in 3months System own.Own PS3 but no 360Discard no-PS3 and own-360 OtherMost have finished LBPDiscard non-LBP completers

57. 4. Moderation 57

58. Moderation = Craft 58

59.  Keep participants comfortable  Ask non-leading and open questions  Give users the voice but lead them where you want  Resist the urge to help  Stay objective, impartial and relaxed  If you make a mistake, carry on Some tips 59

60. 60

61. 5. Observation 61

62. Note taking moderator Observer/ Note-taker Outsource!

63. o Write down in advance what you want to observe o Tally each time a user experiences the issue, follow up o Focus on what happens and why, no just what participants say Note-taking Tips

64. 6. Analysis 64

65. analysing data G ather A ggregate P rioritise S ort G ather A ggregate

66. Combining data Secret recipe

67. 7. Reporting 67

68. Workshop solutions 68 Ensure your research has the maximum impact

69. Write a full report 69 The final report A document to refer to A list of issues to follow up on

70. Re-testing 70 TestFind Issues Team introduces fixes Did it work?

72. Games User Research Special Interest Group: http://www.linkedin.com/groups?mostPopular=&gid=1873014 http://www.linkedin.com/groups?mostPopular=&gid=1873014 Games User Research Summit: http://emiie.iat.sfu.ca/gur/ http://emiie.iat.sfu.ca/gur/ Game User Research Library: http://sites.google.com/site/gamesuserresearch/ http://sites.google.com/site/gamesuserresearch/

73. Part 2 - Activity 73

74. 1. 5 groups 2. Find a game you know o Check on your phone / browser history 3. Discuss profile(s) of users 4. Kick off – talk to the producer (i.e. me) 5. Setup a research plan o Method, moderation guide, questionnaires… 6. Write a proposal on a poster (pitch) 7. Present / discuss Activity 74 30 min

75. Questions? Email: cyril_rebetez@scee.net Thank You! 75 Our Mailing List - Helping with research - Job opportunities - Training  in central London… http://tinyurl.com/SonyUserResearch