Course overview and structure Dates: May 23-June 20, no class May 28 Times: 5:30-8:30pm ET + team meetings What to do if missing all or part of class? Meet in person: for class session, separately, or not at all? Twitter and other social media Social media survey Online course feedback: you are not guinea pigs! Questions: cell: cell Digital Strategies for Health Communication Evaluation Lisa Gualtieri, PhD, ScM Tufts University School of Medicine July 24, 2014

Mobile First Designing for mobile first instead of retrofitting existing practices into mobile format 2

Base: 2,116 US online adults who own a mobile phone; Base: 549 US online adults who own a tablet Source: North American Technographics Telecom And Devices Online Recontact Survey, Q (US) Where are mobile phones and tablets used?

Search to apps 4

What do mobile devices provide health seekers? Immediacy and access Affinity Multiple methods of input/output Context

Top 10 health searches 2011 Web 1. Cancer 2. Diabetes 3. Symptom 4. Pain 5. Weight 6. Infection 7. Virus 8. Diet 9. Thyroid 10. Sleep Healthline Networks Mobile 1. Chlamydia 2. Bipolar disorder 3. Depression 4. Smoking/quit smoking 5. Herpes 6. Gout 7. Scabies 8. Multiple Sclerosis 9. Pregnancy 10. Vitamin A

How to develop an evaluation plan Evaluate to increase success at achieving your goals Evaluate as often as possible from conception to launch Gather data post-launch Techniques to evaluate – Expert reviews, heuristic evaluation, and formative evaluation The stages at which you will conduct evaluations and what will happen For each evaluation: – What are your goals pertaining to appeal, usability, and effectiveness – The number and demographics of the people you will test and how you will recruit and compensate them – Questions to ask 7

Expert reviews Expert reviews are conducted by “experts” from multiple perspectives and focus on the site using the knowledge of the expert in a domain Who are experts? 8

Heuristic evaluations Heuristic evaluations use a set of heuristics or a checklist – Does the site meet certain usability criteria – Does the site meet the stated purpose 9

10 Formative evaluation Formative evaluation is a type of usability evaluation that helps to "form" the design for a product or service Purpose of formative evaluation is to obtain user feedback during the design and development stages of a project Feedback is generally in three areas: appeal, usability, and effectiveness at accomplishing a task Formative evaluation can be conducted on concepts, paper prototypes, screen mock-ups, a working prototype, or an alpha release, with the goal of using the feedback to improve the design at the earliest possible stages when the cost of making changes is lower

Formative evaluation: receive feedback early when easier to change Use incentivized subjects close to target audience Guide through scripted tasks using a think aloud protocol – What do you expect…? – What are your first impressions when you see the screen? – Where do you want to start and what do you expect to happen? Hardest to provide context and measure effectiveness at achieving goal Formative evaluation process start Script sessions Conduct sessions Compile results Review and prioritize Redesign Issues detected Develop

Context Real or simulated context – Environmental – interruptions, movement, noise, multitasking, etc. – Psychological—motivation, emotional barriers, level of engagement – Usability—appealing, easy to accomplish tasks, develop new ones Techniques – Field tests – Participatory design – Market response 12