Presentation is loading. Please wait.

Presentation is loading. Please wait.

Veronica Zammitto Outline:  Game User Experience Evaluations  Case Study 1: NBA Live 10  Case Study 2: NHL 11  Take Away  Q&A.

Similar presentations


Presentation on theme: "Veronica Zammitto Outline:  Game User Experience Evaluations  Case Study 1: NBA Live 10  Case Study 2: NHL 11  Take Away  Q&A."— Presentation transcript:

1

2 Veronica Zammitto Outline:  Game User Experience Evaluations  Case Study 1: NBA Live 10  Case Study 2: NHL 11  Take Away  Q&A

3 Veronica Zammitto Game User Experience Evaluation Common Current TechniquesNew Techniques Qualitative Methods: Interview Focus Groups Think Aloud Survey Subjective information interpreted by an expert. Answers “Why” questions. Quantitative Methods: Psychophysiological signals (‘biometrics’) Eye Tracking Telemetry Measureable, objective, continuous information. Answers “What” questions

4 Veronica Zammitto Mixed Method Approach for Evaluating Sports Games  Triangulating: Interviews UX Eye Tracking Psychophysiology Telemetry Survey In-depth understanding of User Experience (UX). Support design decisions.

5 Veronica Zammitto

6

7

8 Case Study 1: NBA Live10  NBA gameplay issues  to identify:  successful and unsuccessful gameplay aspects.  emotional profile of the player  engagement and emotions  attentional focus  UX throughout the game, and for certain events.

9 Veronica Zammitto Eye Tracking  Hardware + software  X, Y on screen  Tracking users’ gaze can reveal the player’s focus.  DIYS, ~US$ 4,000 to 80,000

10 Veronica Zammitto

11 Eye Tracking  By using ET we can identify where players’ attention is.  Fixation  Saccades  Gaze Movement  Patterns

12 Veronica Zammitto

13

14

15

16 Play Styles video

17 Veronica Zammitto Triangulating Eye Tracking & Survey

18 Veronica Zammitto Telemetry  Hooks in the game engine that flag and time stamp pre-defined events.  Players’ in-game behavior  Statistical analysis  Visualizations  Machine learning algorithms Lay Up Dunk Pass Call Time Out Switch Players Steal

19 Veronica Zammitto Events Performed by Players Through Time in NBA 10

20 Veronica Zammitto Events Performed by Players Through Time in NBA 10

21 Veronica Zammitto Passes sent by players

22 Veronica Zammitto Players’ Scoring Location 91.4 % of the shots 58.6 %

23 Veronica Zammitto AI Scoring Location 75 %

24 Veronica Zammitto What Went Right – NBA study  Better understanding of :  players styles and demographics.  In-game behavior  Identification of emotions (next slides)  The new techniques were proven to provide useful data to development.  Rethink the role of game elements. I.e., coach.  Create tutorials for court observation based on eye tracking data  Worthy of further investment to continue with studies

25 Veronica Zammitto What Went Wrong  Large scope of the NBA study  Impacted synchronization of the usability study with production’s delivery schedule.  The study should have been subdivided into mini assessments to achieve a quicker turn around.  Low involvement of production in the project.  Manual coding:  Time consuming.

26 Veronica Zammitto Case Study 2: NHL 11  Same techniques used for NBA  Adjustments from lessons learnt:  Narrower focus: “Game Presentation”  Front-End Visualizations (Overlays).  I.e.: Do players look at information provided in the UI?  Cut Scenes (NIS):  Are they watched or skipped?  High involvement with the development team  Meetings with development for a ‘statement of work’.  Iterative process with development  Helps to define the root of usability questions.

27 Veronica Zammitto Front End: Overlays

28 Veronica Zammitto Scoreboard Faceoff Overlay Offside Warning Icing Warning Penalty Overlay Line Change- Strategy Player Overlay Line Change- Strategy AI Overlay Fight-Controls Fight-Stamina Player Fight-Stamina AI Player’s Name Overlay Scouting Report End of Period Stats

29 Veronica Zammitto Scoreboard Faceoff Overlay Offside Warning Icing Warning Penalty Overlay Line Change- Strategy Player Overlay Line Change- Strategy AI Overlay Fight-Controls Fight-Stamina Player Fight-Stamina AI Player’s Name Overlay Scouting Report End of Period Stats

30 Veronica Zammitto Scoreboard Faceoff Overlay Offside Warning Icing Warning Penalty Overlay Line Change- Strategy Player Overlay Line Change- Strategy AI Overlay Fight-Controls Fight-Stamina Player Fight-Stamina AI Player’s Name Overlay Scouting Report End of Period Stats

31 Veronica Zammitto Scoreboard Faceoff Overlay Offside Warning Icing Warning Penalty Overlay Line Change- Strategy Player Overlay Line Change- Strategy AI Overlay Fight-Controls Fight-Stamina Player Fight-Stamina AI Player’s Name Overlay Scouting Report End of Period Stats

32 Veronica Zammitto Overlays Distribution FrequencyPercent 1st Period nd Period rd Period OT381.6 Total FrequencyPercent Fight14.6 Gameplay NIS Total

33 Veronica Zammitto Percent Ignored75.7 Observed24.3 Total100.0 Only ¼ of the overlays are actually observed, the other 75% are ignored.

34 Veronica Zammitto Percentage of Observed and Ignored Overlays during different game sections. Quality and sensitive information that helps the player has more chances to be looked during NISes.

35 Veronica Zammitto Overlay “Map” in NHL 11 with Observed and Ignored proportions

36 Veronica Zammitto

37 How people observed the overlays. Overlay Observation Count (avg. visits per appearance) Observation Length (avg. in seconds per appearance) End of Period Stats Scouting report Faceoff-Overlay Icing Warning Offside Warning Penalty Overlay Pulling Home Goalie Fight Controls Fight Stamina AI Fight Stamina Player Line Change-Strat - AI Line Change-Strat - Player Player's Name - Overlay

38 Veronica Zammitto Non-Interactive Sequence (NIS) NISes Scripts - Subsets Event

39 Veronica Zammitto NIS’ Map NISPercentage% watched% skipped Canned Context Sensitive Replay

40 Veronica Zammitto NISes by event and type

41 Veronica Zammitto The effectiveness of NIS is a combination of its type (Canned, Context Sensitive, and Replay), the event that triggers the sequence, and its frequency.

42 Veronica Zammitto Psychophysiology (Biometrics)  Infer emotions from physiological data.  Arousal: engagement, excitement, magnitude of emotions.  Valence: positive (fun) or negative (frustration) Emotional Valence arousal FrustratingExciting Boring Fun

43 Veronica Zammitto Galvanic Skin Response (GSR)  Psychological arousal.  Skin’s conductance increases when a person becomes excited, stressed, or anxious.

44 Veronica Zammitto

45 Electromyography (EMG)  Emotional valence.  Sensors capture and amplify muscles’ contractions.  Facial muscles:  Smiling (zygomatic) = Positive emotions  Frowning (corrugator) = Negative emotions (or) Cognitive load

46 Veronica Zammitto Facial EMG

47 Veronica Zammitto

48 Player’s positive emotional reaction when scoring in NBA Live 10

49 Veronica Zammitto Isomorphism between telemetry and GSR

50 Veronica Zammitto Emotional Profiling of NHL 11 Arousal = exciting, engagement.

51 Veronica Zammitto Levels of Arousal in NHL 11: Tendency of increasing arousal throughout the game.

52 Veronica Zammitto Player 2 vs AI 0 “I had a good game, but I should have readjusted the AI level. To me it’s also a challenge to score a lot of goals when I know I’m going to win, then it’s how many. So, you established your own achievement. Yes, I wanted to get 6 at least by the end of the game.” Player 6 vs AI 2

53 Veronica Zammitto NHL’s Arousal Profile NHL 11: Arousal profile

54 Veronica Zammitto Arousal-Valence Space Arousal = excitement Valence =  positive emotions  Negative emotions

55 Veronica Zammitto

56

57

58

59 Take Away  New possibilities in user experience, mixed method with:  Identifying the concrete information, cross-references.  Information that supports design decisions. Eye-tracking Biometrics Telemetry Interview Survey Emotions Engagement Attention In-game behavior

60 Veronica Zammitto Thanks! Questions?


Download ppt "Veronica Zammitto Outline:  Game User Experience Evaluations  Case Study 1: NBA Live 10  Case Study 2: NHL 11  Take Away  Q&A."

Similar presentations


Ads by Google