Computing for Social Needs Jennifer Mankoff UC Berkeley

Inspiration in the World: Finding the Right Combination  Hard, real problems  Hard HCI problems  Low intuition about users  Success hard to test  Technology not always a good solution  Hard computer science problems

Examples  Word prediction (+ & -)  Augmented canes (+ & -)

Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

Approaches To Research in CSN  Design: For users  Method: For designers/evaluators  Tool: For programmers/designers

Approaches To Research in CSN  Design: For users  Identify need  Investigate solutions  Prototype, test & iterate  Method: For designers/evaluators  Tool: For programmers/designers

Approaches To Research in CSN  Design: For users  Method: For designers/evaluators  Identify model or theory  Test against circumstances or population  Iterate  Tool: For programmers/designers

Approaches To Research in CSN  Design: For users  Method: For designers/evaluators  Tool: For programmers/designers  Identify repeating need or use of technology  Abstract out  Test for reusability

Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

Design Example: Nutrition  Need: Healthier diets  Assumptions  Idea: Keep track of purchases, display advice

Design Example: Nutrition  Need: Healthier diets  Manage disease  America’s weight problem  Manage child health  Assumptions  Idea: Keep track of purchases, display advice

Design Example: Nutrition  Need: Healthier diets  Assumptions  People don’t really know what they consume  Receipts contain enough information for us to estimate nutrition  Idea: Keep track of purchases, display advice

Design Example: Nutrition  Need: Healthier diets  Assumptions  Idea: Keep track of purchases, display advice

Nutrition: Hard HCI Problems  Formative evaluation: testing perception  Interface design  Summative evaluation in real-use setting

Nutrition: Formative Eval  Survey shoppers  Background research

Nutrition: Formative Eval  Survey shoppers  Perceived calcium consumption  Perceived need for supplements  Calcium consumption in receipts  Background research

Nutrition: Formative Eval  Survey shoppers  Background research  Use of shopping receipts in bookkeeping  Interest in nutrition  % of time eating out  Impact of coupons, advice on shopping behavior

Nutrition: Hard HCI Problems  Formative evaluation: testing perception  Interface design  While at home  Continual  Peripheral  While shopping  While entering data  Summative evaluation in real-use setting “Was that ‘Apple cider’ Or ‘Apple scraper’

Nutrition: Hard HCI Problems  Formative evaluation: testing perception  Interface design  Summative evaluation in real-use setting  Measures change in awareness  Measures change in behavior

Nutrition: Hard Computer Science Problems  Recognition  OCR  Who eats what  Quantities, ingredients  Ambiguity

Nutrition: Hard Computer Science Problems  Recognition  OCR  Who eats what  Quantities, ingredients  Ambiguity

Nutrition: Hard Computer Science Problems  Recognition  Ambiguity  Resolving imperfect recognition automatically  Resolving imperfect recognition with user’s help

Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

Method Example: Comparative Accessibility  Need: Increased accessibility in all interfaces  Assumptions  Idea: Develop metrics for interpreting simulated testing results

Method Example: Comparative Accessibility  Need: Increased accessibility in all interfaces  More inclusive  Increase quality of life  Assumptions  Idea: Develop metrics for interpreting simulated testing results

Method Example: Comparative Accessibility  Need: Increased accessibility in all interfaces  Assumptions  Can’t test every interface with every type of disability  Can simulate disability sufficiently for testing  Idea: Develop metrics for interpreting simulated testing results

Method Example: Comparative Accessibility  Need: Increased accessibility in all interfaces  Assumptions  Idea: Develop metrics for interpreting simulated testing results

Comparative Accessibility: Hard HCI Problems  Can a novice simulating disability give feedback on an interface designed for experts in that disability?  How should heuristics include accessibility?  How do disabilities impact GOMS models?  How do disabilities impact Fitts’ law?

Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

Tool Example: Mouse predictions  Need: Access to any application  Assumptions  Idea: Recognize problems, predict targets, and use that to make the mouse do the right thing

Tool Example: Mouse predictions  Need: Access to any application  Equal access  Increased independence  Assumptions  Idea: Recognize problems, predict targets, and use that to make the mouse do the right thing

Tool Example: Mouse predictions  Need: Access to any application  Assumptions  Low vision or motor impairment  No access to application code  Access to OS (e.g. app can be installed)  Idea: Recognize problems, predict targets, and use that to make the mouse do the right thing

Tool Example: Mouse predictions  Need: Access to any application  Assumptions  Idea: Recognize problems, predict targets, and use that to make the mouse do the right thing.

Mouse predictions: Hard HCI problems  Existing motion models only account for averages  Existing user models inaccurate  UI for compensation unclear

Mouse predictions: Hard HCI problems  Existing motion models only account for averages  Minimum jerk model: X(t) = X 0 + (X 0 – X f ) (15    3 )  Fitts’ law: MT = a + b log(A/W)  Existing user models inaccurate  UI for compensation unclear

Mouse predictions: Hard HCI problems  Existing motion models only account for averages  Existing user models inaccurate  KLM  extra cognitive cycles  No model of fatigue  UI for compensation unclear

Mouse Prediction: Other Models  Velocity  Thrashing ( = target)  Spasming  Overshooting  Other characteristics?

Mouse predictions: Hard HCI problems  Existing motion models only account for averages  Existing user models inaccurate  UI for compensation unclear  “Beat Fitts’ law”  Feedback affects recognition

Mouse Predictions – UIs for Compensation  Gravity wells and area mouse  Mediation

Mouse Predictions: Hard Computer Science Problems  Recognition  Account for feedback  Account for fatigue  Ambiguity  Better interfaces for multiple targets?  Interface for multiple directions?  Appropriate balance of control and automation

Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

Conclusions  Plenty of hard real problems  Plenty of hard HCI problems  Plenty of hard computer science problems  Research needed in designs, methods & tools

Thank You For More Information:

Tool Example: Reconstruction of Mismatched Interfaces  Need: Adaptation to any set of input devices