1. Rationalising and Designing Methodologies for New Media Research Gosia Kwiatkowska gosia@uel.ac.uk

2. Designing New Media Choosing your research method

3. Quantitative or Qualitative Method or both?

4. Quantitative Methods – Designed to ensure objectivity, reliability and ability to generalise – Test predetermined hypothesis – explanations – Who, how much, how many? – Closed questions – Short answers – Averages, percentages, ranges, means, frequencies – Can generate accurate and precise data – Can test statistical relationship between variable – Can prove whether or not a particular problem exists – Can identify specific characteristics of a population http://documents.wfp.org/stellent/groups/public/documents/ko/mekb_module_13.pdf

5. Quantitative Methods Strengths – Robust – Objective – Verifiable Weaknesses – Out of context – human behaviour, real world settings are not considered – Any variables left out of data collection are not used in analysis

6. Qualitative Method? – User perspectives – Questions: how and why? – Captures perceptions, judgements, meanings, processes and reasons – Open ended questions, checklist of topics – Hypothesis and follow up questions are generated during data collection and analysis – Subjective – not easy to objectively verify – Interviews allow to probe – Provides deeper insight – Broader understanding and explanations, views http://documents.wfp.org/stellent/groups/public/documents/ko/mekb_module_13.pdf

7. Qualitative Method? Strengths – Participatory – Rich, detailed data – Considers users perspectives and the context for their behaviours Weaknesses – Hard work – Time consuming – Smaller sample of users – Not easily verifiable – Not easy to group your responses/categorised

8. Quantitative or Qualitative or Both? Quantitative methods can highlight an issues that could then be studies in depth using qualitative methods. Qualitative methods might be used at the beginning of a study to help the researcher to decide what closed questions could be used in the bigger quantitative survey or questionnaire.

9. Design phases Phase 1 - User analysis – Questionnaires/ethnography/interviews Phase 2 - Conceptual design – Cards sorts/cognitive walkthrough – flowcharts Phase 3 - Testing and implementation – User testing of prototypes – Final design

10. Phase One- User Analysis http://www.grameenfoundation.applab.org/challenges/what-innovative-mobile-money-mm-products-can-we-develop-that-are- suitable-for-poor-unbanked-customers-and-viable-for-our-commercial-partners/timeline.html

11. Tools Surveys: type of questions? Interviews: interview questions? Focus groups: purpose and objectives, discussion notes, who, why, where? What did you find out? Which quotes, comments will you use to support your findings? Observations: observation checklist Case studies: use triangulation

12. Checklist for designing and conducting interviews, focus groups, surveys Decide what you need to know. State why you need to know this information. What is the best way of obtaining this information. Outline your questions. Check wording of each question. Decide on the tool for data collection and the question type. Refine the questions and order them. Consider how questions will be analysed.

13. Checklist for designing and conducting interviews, focus groups, surveys cnt Write instruction that might be included in your surveys. Consider layout and appearance. Decide on your sample, are they representative. Prepare a schedule. Pilot. Revise, if necessary. Do your best to avoid bias. Select who to interview. Ensure official channels have been cleared. Information sheets, consents etc Introduce yourself and explain the purpose of the research

14. Checklist for designing and conducting interviews, focus groups, surveys cnt

15. Designing your questionnaires Questions types – List – Category – Ranking – Scale – Quantity – Grid – Verbal

16. List

17. Summary sheet Question 13: What do you use the internet for? Finding interesting information Finding pictures Finding and watching videos Finding and listening to music Online learning ShoppingOnline gaming Other || ||||||||| || | ||| 171812211827263

18. Category questions How old are you? ______ What will you do with the responses? Depends on what you want to know? e.g. Average age of your users? If so, what sort of average or measure of central tendency will suit your purpose: the arithmetic means, the median, or the mode?

19. The arithmetic mean Arithmetic mean = sum or each value/total number of values. 26 participants Ages: 21,22,22,25,25,25,25,25,25,26,26, 26, 27, 28, 29, 30,30,32,32,33,34,35,35,35,35,35 Add ages together = 743 Divide 743 by 26 The mean = 28.57

20. Mean Group A Ages: 21,22,22,25,25,25,25,25,25,26,26, 26, 27, 28, 29, 30,30,32,32,33,34,35,35,35,35,35 Group B Ages: 18,18, 18, 19, 19,19, 19, 20, 20, 20, 21, 21, 22,23,24,25,26,29,30,39,39, 43, 45, 49, 52,65, The same mean but clear difference between the dispersion of results

21. The median Allows us to find the middle value If we have an odd number of values If we have an even number of values: take the average of the middle values Group A Ages: 21,22,22,25,25,25,25,25,25,26,26, 26,27,28, 29, 30,30,32,32,33,34,35,35,35,35,35 Median=(27+28)/2 = 27.5 Group B Ages: 18,18, 18, 19, 19,19, 19, 20, 20, 20, 21, 21, 22,23,24,25,26,29,30,39,39, 43, 45, 49, 52,65 Median =22.5 You decide which one gives more realistic picture.

22. The mode The most frequently occurring value Group A Ages: 21,22,22,25,25,25,25,25,25,26,26, 26,27,28, 29, 30,30,32,32,33,34,35,35,35,35,35 Modal score = 25 Group B Ages: 18,18, 18, 19, 19,19, 19, 20, 20, 20, 21, 21, 22,23,24,25,26,29,30,39,39, 43, 45, 49, 52,65 Modal score = 19

23. Range Range the difference between the highest and the lowest values measured Group A 21-35 range = 35-21 = 14 Group B 18-65 range = 65-18 = 47 Problem? What about the middle numbers?

24. Interquartile range Interquartile range indicates the extent to which data vary. Derived from the median, the highest and the lowest quarter of the measures are omitted and the interquartile range of the middle 50 % of values is quoted Group A 21,22,22,25,25,25,25,25,25,26,26, 26,27, median point 28, 29, 30,30,32,32,33,34,35,35,35,35,35 Interquartile range is 25-32 or 7 years

25. Alternative question How old are you? 18-2930-3940-49 50-59 60+ You can still calculate the arithmetic means by using the mid point of each class interval.

26. Scales

27. Verbal (open) question

28. Recording your data Data must be recorded, analysed and interpreted. Identify categories. Prepare summary sheets. Experiment with different ways of presenting findings. Remember all data require interpretation.

29. Presenting your data and analysis

30. Phase Two-Conceptual design  Making sense of data  Organise data  Aggregation (combine/collapse redundant pieces of data)  Juxtaposing related pieces of data (putting together)  Naming the resulting data clusters  Juxtaposing unrelated pieces of data  Exploring alternative solutions  Choosing a design (Safer, 2010, p.95)

31. Conceptual design-Tools – Brainstorming – Card sort – Personas – Scenarios – Flowcharts – Use cases

32. Brainstorming

33. Card Sort Step 1 – separate all information Step 2 – organise the cards into related areas Step 3 – label your groups What if one topic fits into more than one category? Test This will provide basis for your information architecture, the structure and the design.

34. Card Sort Advantages Can be done before design Quick and easy Good way of finding out underlying structures for your design Good way of finding out how the end user organises information Disadvantages Difficult to navigate when you have lots of categories Limited to the topics that you put on the cards Card sort outcomes suggest structures

35. Personas

36. http://geoinformatics.fsv.cvut.cz/gwiki/Developing_web_map_application_based_on _user_centered_design

37. From Personas to Cognitive Walkthroughs Scenarios: typical task, goal, conditions at the beginning of the task, activities in which the persona will engage, the outcomes of these activities.

38. Scenario http://csweb.cs.bgsu.edu/maner/domains/Scenario.htm

39. Flowcharts Information architecture Organise information Show navigational structure Efficiency Effectiveness It should correspond to: Content User needs Our objectives Garrett, 2010, p94

40. Use cases

41. Phase Three-Testing and Implementation The physical design: – Look – Components – Layout Tools: – Low-fidelity prototypes – Evaluations – Wireframes – High-fidelity prototypes

42. Low-fidelity prototyping Nielsen’s two types of prototypes – Horizontal – Vertical http://www.useit.com/papers/guerrilla_hci.html

43. Low-fidelity prototypes: Easy and cheap to make Flexible-can be changed and re-arranged Complete enough to give useful feedback about specific design question/s http://www.nngroup.com/reports/prototyping/video_stills.html Nielsen Norman Group

44. Things to consider in order to define the scope of the prototype and the focus: What feedback do you need at this point in the design process? How much of the design should you prototype? Should you cover all of the areas but without great detail (breadth vs. depth)? Should you cover one area in great detail? Nielsen Norman Group

45. Evaluation Begin evaluations early in the design process. Evaluation is an integral part of the development process and can take the form of an informal walkthrough or a more structured heuristic evaluation. Formal usability testing can begin once a prototype has been developed.

46. Wireframe-blueprint of a product

47. Wireframes cnt Wireframes help to create template layouts that can be used to impose a consistent structure throughout the interface

48. http://www.gliffy.com/examples/wireframes/

49. Service blueprint Similar like wireframe for digital product, service blueprint presents service moments and service string. E.g. car wash Brainstorm possible service moments Sketch/photograph/single storyboard frame each moment How each service moment affects service elements (the environment, objects, process, and people involved)

50. Service string-Scenario Storyboard-that shows the pathway through the service. This should show the whole picture of the service-from seeing the sign for the service to exit.

51. Saffer, 2010, p 156

52. Cognitive walkthroughs Follow the scenario using the flowcharts or paper prototype. Questions to ask: – Will the user realistically be trying to do this action? – Is the control for the action visible? – Is there a strong link between the control and the action? – Is feedback appropriate?

53. From Low to High Fidelity prototypes High fidelity prototypes are interactive prototypes that represent various degrees of functionality http://www.grameenfoundation.applab.org/challenges/what-innovative-mobile-money-mm-products-can-we- develop-that-are-suitable-for-poor-unbanked-customers-and-viable-for-our-commercial- partners/timeline.html

54. Nielson’s Heuristics ‘Rules of Thumb’ 1.Visibility of system status 2.Match between system and the real world 3.User control and freedom 4.Consistency and standards 5.Error prevention 6.Recognition rather than recall 7.Flexibility and efficiency of use 8.Aesthetic and minimalist design 9.Help users recognize, diagnose, and recover from errors 10.Help and documentation 11. http://www.useit.com/papers/heuristic/heuristic_list.html

55. Heuristics Process http://uxcentered.wordpress.com/2009/12/27/hello-world/

56. Heuristics Experts Set of criteria Scenario where each step is tested against the heuristic criteria. Report

57. http://uxcentered.wordpress.com/2009/12/27/hello-world/

58. Final design

59. References Bell, J. (1999) Doing your research project: A guide for first-time researchers in education and social science. (3 rd Ed) Open University Press, Buckingham, Philadelphia Garrett, J. (2010) The elements of user experience: user-centred design for the web and beyond. New Riders Press, Barkley, CA Goto, K. & Cotler, E. (2002). Web ReDesign. Workflow that Works. Indianapolis: New Riders. Heim, S. (2008) The Resonant Interface HCI Foundations for Interaction Design, Pearson Addison-Wesley, New York Norman, D. (2004). Emotional Design. New York: Basic Books. Preece, J., Rogers, Y. & Sharp, H. (2002). Interaction Design: Beyond Human-Computer Interaction. New Jersey: John Wiley & Sons. Saffer, D. (2010)Designing for interaction. (2 nd ed) New Riders Press, Berkley, CA Shneiderman, B. and Plaisant, C. (2005). Designing the User Interface: Strategies for Effective Human-Computer Interaction (Fourth Edition) Pearson Addison Wesley, Boston, MA. Torres, R.J. (2002). Practitioner's Handbook for User Interface Design in Development. New Jersey: Prentice Hall. Van Duyne, D., Landay, J. & Hong, J (2003). The Design of Sites. New York. Addison- Wesley. www.useit.com/papers/heuristic/heuristic_list.html www.cse.unr.edu/~dascalus/ID2011_11.ppt http://uxcentered.wordpress.com/2009/12/27/hello-world/ http://www.grameenfoundation.applab.org http://www.gliffy.com/examples/wireframes http://www.nngroup.com/reports/prototyping/video_stills.html http://www.userfocus.co.uk/articles/cogwalk.html