1. Rich GuntherJeff JanisScott Butler The UCD Decision Tool www.ovostudios.com/upa2001

2. Abstract Present background and results from a survey of 100 usability practitioners Identify trends in the state-of-the- practice Introduce you to an online summary of data you can use (www.ovostudios.com/upa2001). Gunther, Janis, & Butler – UPA 2001

3. Audience Expectations What methods should I learn first? How can I be more effective during development? How can I better demonstrate return on investment? How can I structure and position my team? How can I influence …you sell UCD and what push-back can you expect? Gunther, Janis, & Butler – UPA 2001

4. Inspiration for This Presentation Recognized the wealth of knowledge that lies latent in our community of practitioners Wanted to learn how practitioners address day-to-day challenges: –where we position UCD Tools and Processes within the Product Development Lifecycle –how we communicate with our internal and external customers, build teams, and structure processes –how this varies based on demographic information Gunther, Janis, & Butler – UPA 2001

5. This Presentation Is… An opportunity to learn how 100 of your colleagues structure their teams and services A discussion of the interaction between product development lifecycles and user- centered design A forum for usability practitioners to talk about their successful and not-so-successful experiences engaging with development teams. Gunther, Janis, & Butler – UPA 2001

6. This Presentation Is Not… An in-depth tutorial on UCD methodologies A criticism of: –Consultants –Corporate practitioners –Marketing departments –Developers –Management Gunther, Janis, & Butler – UPA 2001

7. Usability’s Challenges Gunther, Janis, & Butler – UPA 2001

8. The Lessons of the Past “The way in which we deliver the message must be pointed and unambiguous… we must deliver punch and clarity”. (Kreitzberg and Shneiderman) “Competing claims about usability, …ambiguous terminology,…competing paradigms,… and limited audience interest” (Paul McInerney) We need to engage with development teams from the beginning through the end of design. (Darnell and Halgren) Gunther, Janis, & Butler – UPA 2001

9. Examples from Practitioners “I’m looking for a consultant who is not just going to usability test the product to death.” “I don’t know why Jakob says to do this… ask him.” “Yeah, our company wants to do usability.” Gunther, Janis, & Butler – UPA 2001

10. Examples from Developers “You guys make our products easier to use… what else do you do?” “That usability team! All they care about is our customers!” “Did you make this up, because if a user said that, then they don’t know what they are talking about!” Gunther, Janis, & Butler – UPA 2001

11. Example from Dilbert, 1 (Insert bridge here.) Gunther, Janis, & Butler – UPA 2001

12. Example from Dilbert, 2 (Insert bridge here.) Gunther, Janis, & Butler – UPA 2001

13. Audience Experience What challenges can audience members add? Gunther, Janis, & Butler – UPA 2001

14. So, What Are We Doing about All This? Lessons of the Past: are we following them? Practitioner Quotes: what are we doing to adjust the state-of-the-practice (if it needs adjusting)? Developer Quotes: what are we doing to correct the misperceptions? Dilbert: if comics reflect our dynamics, there must be a problem. Gunther, Janis, & Butler – UPA 2001

15. Methodology Gunther, Janis, & Butler – UPA 2001

16. Goal of the Survey By surveying 100 usability practitioners about their team composition and mission, company demographic, and the successes and failures they had implementing UCD processes, we hoped to get an idea of which variables influenced their ultimate level of success or failure. Gunther, Janis, & Butler – UPA 2001

17. Question mapping Company demographic – size, dev type

18. Survey Collection Method Constructed 10-question internet survey with single-selection, multi-selection, and free form text input Requested participation from UTEST, HFES, SIGCHI, and the STC mailing lists Posted the survey for roughly two weeks Took it offline when responses reached the 100-response mark. Gunther, Janis, & Butler – UPA 2001

19. Analysis Method Collated responses into an Excel spreadsheet Calculated overall results for each question Personalized results by organization size, practitioner type, team size, and team tenure Investigated trends in mission statements, UCD method positioning, and follow-up activities. Gunther, Janis, & Butler – UPA 2001

20. Results Gunther, Janis, & Butler – UPA 2001

21. Respondent Profile The following profile information is presented for respondents: 1.Organization type 2.Company size 3.Usability group tenure 4.Team size Gunther, Janis, & Butler – UPA 2001

22. Organization Type Most of us work with developers who are internal to our own company. Gunther, Janis, & Butler – UPA 2001

23. Company Size The majority works for companies with less than 3500 employees. Gunther, Janis, & Butler – UPA 2001

24. Usability Group Tenure The majority works in a a team less than 5 years old. Gunther, Janis, & Butler – UPA 2001

25. Team Size Mean = 6.4 Median = 4 Mode = 1 Min = 1 Max = 74 Gunther, Janis, & Butler – UPA 2001

26. Tracks Gunther, Janis, & Butler – UPA 2001

27. Three Tracks Beginner/Intermediate: focus on skill and career development or change Manager: focus on building a team Influencer: focus on usability strategy, influencing, deployment Gunther, Janis, & Butler – UPA 2001

28. Beginner/Intermediate Track 1.What UCD skills should I learn first? 2.What UCD skills should I learn next? 3.What if I want to move beyond usability? Gunther, Janis, & Butler – UPA 2001

29. What UCD Skills Should I Learn First? Respondents indicated the following activities were their most successful: –Usability testing (39%) –Paper or other prototyping (19%) –Heuristic evaluation (10%). Best selling activities across the development lifecycle were: –Customer interviews (Ph.1, 46%) –Paper or other prototyping (Ph.2, 65%) –Usability test (Ph.3, 41% and Ph.4, 36%). Gunther, Janis, & Butler – UPA 2001

30. What UCD Skills Should I Learn Next? More than 60% of respondents: –Usability test (71%) –Paper or other prototypes (65%) –Heuristic review (65%) More than 40% of respondents: –Cust Intvws (56%) –GUI Spec’s (47%) –Use cases (46%) –UCD Sessions (45%) –Vision/Scope documents (41%) Gunther, Janis, & Butler – UPA 2001

31. Moving Beyond Usability Let’s say you want to re-tool for another role on your team. What roles are most common? –Usability specialist (86%) –UI designer (61%) –Developer (33%) –Information developer (32%) –Graphic artist (21%) Gunther, Janis, & Butler – UPA 2001

32. Questions? Gunther, Janis, & Butler – UPA 2001

33. Manager Track 1.What is a typical mission for a team like mine? 2.Who should be on my team? 3.What UCD activities should we offer? 4.What processes and procedures should we have in place? Gunther, Janis, & Butler – UPA 2001

34. What is a Typical Mission for a Team Like Mine? An item analysis of mission statements indicated that: –61% conduct miscellaneous UCD activities –37% impact product design –22% are tasked with making things better. 52% of teams have well-defined missions or team responsibilities. Gunther, Janis, & Butler – UPA 2001

35. Who Should be on My Team? UCD teams have team members in the following roles according to a sample of 100 survey respondents: –Usability specialist (86%) –UI designer (61%) –Developer (33%) –Information developer (32%) –Graphic artist (21%). Gunther, Janis, & Butler – UPA 2001

36. What UCD Activities Should We Offer? Your peers’ most successful activities were: –Usability testing (39%) –Paper or other prototyping (19%) –Heuristic reviews (10%). Over 60% of respondents said their teams performed the following activities: –Usability tests (71%) –Paper prototypes (68%) –Heuristic reviews (65%). Gunther, Janis, & Butler – UPA 2001

37. What Processes & Procedures Do I Need? A majority of respondents had the following documentation in place: –Usability activity deliverables (71%) –Roles and responsibilities (55%) –Usability activity timeframes (53%). A large proportion also had: –Design Standards (45%) –Follow-up expectations (40%). Gunther, Janis, & Butler – UPA 2001

38. Audience Brainstorm Any other manager issues? Gunther, Janis, & Butler – UPA 2001

39. Influencers Track 1.What level of developer resistance is typical? 2.What kind of developer resistance can I expect in different phases of engagement? 3.What development lifecycle phases can I sell into? 4.Does life get easier if we engage early in the lifecycle? 5.How can I demonstrate to management that my group is impacting product? Gunther, Janis, & Butler – UPA 2001

40. What Level of Developer Resistance is Typical? Gunther, Janis, & Butler – UPA 2001 83 respondents reported resistance of some type during interactions with developers. 4 respondents explicitly said that they had no problems.

41. What Kind of Resistance Can I Expect? Gunther, Janis, & Butler – UPA 2001 When engaging –General resistance –Usability unawareness. When working with –General resistance –Usability unawareness. When following-up –General resistance –Implementation challenges.

42. What Development Lifecycle Phases Can I Sell Into? All of them. Best SellerRunner-Up Req’tsCust.InterviewUse Cases DesignPaper PrototypeUsability Test Dev’tUsability TestHeur.Review TestUsability TestHeur.Review Gunther, Janis, & Butler – UPA 2001

43. Does Life Get Easier If We Engage Earlier? Problems reported by more than 20% of respondents were: General resistance (engage, work, follow- up) Usability unawareness (engage, work) Implementation challenges (follow-up). Gunther, Janis, & Butler – UPA 2001

44. How Can I Demonstrate Impact on Product? 82 (n) hold follow-up meetings 62 (n) document resolutions made in follow- up meetings 26 (n) track implementation rates (IR). –Report IR to R&D managers on a quarterly basis –Report IR to Quality Management –Input usability defects in our DCR then query for IR. Gunther, Janis, & Butler – UPA 2001

45. Audience Brainstorm Any other influencer issues? Gunther, Janis, & Butler – UPA 2001

46. Web Site Tour Gunther, Janis, & Butler – UPA 2001 www.ovostudios.com/upa2001

47. Wrap-Up Gunther, Janis, & Butler – UPA 2001

48. Suggestions Use the principle of “punch and clarity” to guide activities from engagement, through working with, and following-up. Counteract “competing paradigms” with clear mission statements and procedures. Learn new UCD methods so you can engage throughout the development lifecycle. Gunther, Janis, & Butler – UPA 2001

49. Future Directions - Teams with a clearly defined mission statement, as well as documented roles and responsibilities, will have significantly less trouble engaging with developers. - Teams who position their UCD methods appropriately in the product development lifecycle will have significantly less trouble working with developers. - Teams that track the implementation of design recommendations and document the resolution of these recommendations will have significantly less trouble following up with developers. Gunther, Janis, & Butler – UPA 2001

50. Rich GuntherJeff JanisScott Butler www.ovostudios.com/upa2001