Games User Research Cyril Rebetez User Research Team Sony Computer Entertainment Europe { 1

User Research at Sony PlayStation 2 1. Who are Sony Computer Entertainment? 2. Games user research vs usability testing 3. How to do games user research Recruitment Methods Analysis Reporting (Break) Activities – setup a games user research

Who are Sony Computer Entertainment? 3

4

What does the User Research Team do? … 5 Usability and User Research – During Development

Now is a good time to watch a video 6

What does the User Research Team do? 7

Research Projects What does the User Research Team do? 8

9

Games user research vs Usability? 10

usability effectivenessefficiency memorability learnability satisfaction Productivity

G.U.R effectivenessefficiencyEngagement

Not designed for efficiency 13

Games must challenge 14 (in the right way)

Passionate Audience 15

Different interaction methods 16

Different interaction methods 17

Different interaction methods 18

G.U.R  Reduce Hassle  Maximise productivity  Consistent simplicity  Understanding  Focus on effectiveness & efficiency  Mainly behaviour & some attitude  Reduce interface Hassle  Maximise engagement  Progressive complexity  Understanding & performing  Focus on fun & emotion  Both behaviour & attitude usability

20 Researching User Experience

How to do it? 21

1. Kick off – What are we looking for? 2. Preparation – How do we do it? 3. Recruitment – Who should do it? 4. Moderation – How to make them do it? 5. Observation – What to look for? 6. Analysis – What does it mean? 7. Reporting – How to explain it? How to do it? 22

1. Kick off 23

Objectives 24 User Research kick-off Meet with the people who make the game, What do they want to know? Everything Define the objectives Who are the audience? Decide how we can answer their questions.

2. Methods 25

1 to 1 testing 26

1 to 1 testing 27 Deep testing with 1 user at a time Focus on observation and interviews

1 to 1 testing What questions can it answer? understanding Anything based around understanding, including: Do players know what to do? Do players know where to go? Do players know how to use this weapon or ability? What do players think about a feature? … 28

1 to 1 testing 29 “Make sure the VITA can see the cards”

1 to 1 testing 30 “Make sure the VITA can see the cards”

1 to 1 testing 31

1 to 1 testing Why is it suitable? Allows close observation of the issues Can ask questions to the players 32

Multi-seat testing 33

Multi-seat testing 34

Multi-seat testing 10 users at a time Focus on quantitative data 35

Multi-seat testing What questions can it answer? Are there any difficulty spikes? Can players complete the game? How long does it take players to learn all of the mechanics? And more… 36

RESOGUN 37

Multi-seat testing 38

Multi-seat testing Why is it suitable? Many participants increase reliability Questionnaires can get quantitative data Can test longer sections of gameplay 39

Home Testing Real world setting Uncover issues you can’t find in a lab 40

Home Testing 41

Home Testing 42 ?

Home Testing 43 ?

Home Testing 44

Diary Studies Real world setting Uncover issues that reveal themselves over time 45

Diary Studies 46

Diary Studies 47

Telemetry Interstitial questionnaires RITE Usability Evaluation And more… 48

3. Recruitment 49

Audience

Diversity

 More than 34% of gamers are 18+ females  Less than 18% of gamers are under 17 males  25% of the gamers are 50+ years old  Average age of gamers is 35 years old Diversity

o Use market research outcomes o Define with stakeholders o What is the vision? o Who is likely to play / purchase / use the product? o Is it an established franchise? o Trying to break into a new market? Defining users profiles 53

How many users? 54 “5 users will typically uncover 80% of the usability issues” that the whole target audience will experience (Nielsen, 1998) Usability :

Recruitment Group profile definition  Demographics (age, gender)  Gaming profile: Hardcore, Core, Casual, Fanatic, Kids, etc…)  Type(s) of games played  Hours of play per week (online & offline)  Number of games purchased in the last 3 months  Games & system(s) owned (e.g. 360° + Kinect)  Other criteria (e.g. franchise awareness, play preferences, etc.)

Recruitment Example of screener criterionrequirementscreener Age range(s)5x teens 13 to 16 (equal spread)Discard: 16 yrs old Gender(s)3x boys & 2x girls Gaming habit2 to 5 hrs/week offline & 0 onlineDiscard 5 hrs/week purchaseMax 2 game in last 3 monthsDiscard >2 games in 3months System own.Own PS3 but no 360Discard no-PS3 and own-360 OtherMost have finished LBPDiscard non-LBP completers

4. Moderation 57

Moderation = Craft 58

 Keep participants comfortable  Ask non-leading and open questions  Give users the voice but lead them where you want  Resist the urge to help  Stay objective, impartial and relaxed  If you make a mistake, carry on Some tips 59

60

5. Observation 61

Note taking moderator Observer/ Note-taker Outsource!

o Write down in advance what you want to observe o Tally each time a user experiences the issue, follow up o Focus on what happens and why, no just what participants say Note-taking Tips

6. Analysis 64

analysing data G ather A ggregate P rioritise S ort G ather A ggregate

Combining data Secret recipe

7. Reporting 67

Workshop solutions 68 Ensure your research has the maximum impact

Write a full report 69 The final report A document to refer to A list of issues to follow up on

Re-testing 70 TestFind Issues Team introduces fixes Did it work?

Games User Research Special Interest Group: Games User Research Summit: Game User Research Library:

Part 2 - Activity 73

1. 5 groups 2. Find a game you know o Check on your phone / browser history 3. Discuss profile(s) of users 4. Kick off – talk to the producer (i.e. me) 5. Setup a research plan o Method, moderation guide, questionnaires… 6. Write a proposal on a poster (pitch) 7. Present / discuss Activity min

Questions? Thank You! 75 Our Mailing List - Helping with research - Job opportunities - Training  in central London…