05-773A4: Computer Science Perspectives in HCI, (CS Mini), Spring, 2015, Mini 3 Brad Myers 1. Course overview, topics, and organization; and why hard 1© Brad Myers

2 Course: Course web page: Made accounts for everyone with Firstname.Lastname and your Andrew Make sure you can log in Overview & Schedule: Tuesdays and Thursdays 10:30AM - 11:50AM in GHC 4301 © Brad Myers

3 Instructor Brad Myers Human Computer Interaction Institute Office: Newell-Simon Hall (NSH) 3517 Phone: x Office hours: By appointment or drop by Secretary: Indra Szegedy NSH No TA © Brad Myers

What are we going to do day to day? READING! This is course is centered on the literature You are going to do a LOT of reading And Discussion Started on-line, but mostly in class Which means you need to be here Especially because there are only 12 class meetings 4© Brad Myers

Things you will need to do Every class Read the papers (Yes there are a lot of them) Be here and participate Some of you have commitments driven by your research, so… You get at most 1 excused “absence for professional reasons”  Must be approved in advance (talk to me soon)  For professional reasons such as attending a conference  Still need to do the readings and post your comments (on time) If you are legitimately ill, that’s ok (you will be excused) But beyond the 1st excused absence, you may have to take an incomplete in the course with additional assignments as makeup for the missed work (unexcused absences will just seriously hurt your grade) 5© Brad Myers

Things you will need to do Frequently Paper summaries Posted written summary (starting next time)  Summary by discussion leader (posted by Sunday night for Tuesday’s class, by Tuesday night for Thursday’s class) Presentation Starting for Thursday’s lecture Sign up to be discussion leader of readings by editing course syllabus  Let me know if you can’t edit Each student will do 2 throughout term  At most 1 per topic Comment on at least 2 articles (By noon the day before class) Every class, including the weeks you are presenting, you must comment on at least two of the course readings per lecture (in addition to any summary), even if you are absent for the lecture Once Literature Review  On a topic approved by me based on a 1 page abstract Maybe: Take Home Final Exam 6 © Brad Myers

Presentation responsibilities Online summary of the paper Brief review of the paper in class (remember everyone should have read it!) Provide some intellectual context Where did the ideas come from What ideas build off the ones in your article? Introduce discussion questions 7© Brad Myers

Useful questions for discussion Why is the research question interesting? Do you believe the author’s conclusions? Where the research methods rigorous enough? How does the intellectual or empirical approach compare to others? Online or in class 8© Brad Myers

Papers Generally try to have: 1.A "classic" or "fundamental" article that inspired lots of people, that is, one with a high citation count. 2.A recent article that is really cool. 3.A survey article that discusses lots of systems/approaches, etc. Your feedback on which papers (new / different) Even new / different topics What should be the last topic? 9© Brad Myers

What is this course about? Required “Depth Mini” for the HCII PhD program Purpose: Provide you with some understanding of the literature, methods, and research culture of technically oriented HCI 10© Brad Myers

Choice of Topics History of the field Important perspectives Sampling of important topics From my perspective Similar to last time (2013 – I taught it) Different from previous time’s (Hudson’s) or 2009’s (Mankoff’s) Neither Breadth nor Depth 11© Brad Myers

Content Topics that would be publishable at UIST: significantly novel enabling technologies such as innovative input devices, displays, new interaction techniques, or new media that extend the boundaries of traditional interaction, such as natural user interfaces and interactions, augmented reality, mobile interaction, haptics and tactile feedback interfaces, ubiquitous computing (including wearables), social software, and computer-supported collaborative work innovative user interfaces for difficult interaction contexts or challenging applications. Examples include managing large, complex information sets, usable privacy and security, multi-user interaction, crowdsourcing, fabrication, or techniques that span devices distributed in time and space breakthrough user experiences leveraging techniques such as machine learning, computer vision, computer graphics, speech processing, networking, or human perception and cognition innovative software architectures, design tools, toolkits, programming systems, development environments, tutorial and help systems that support the development and use of the above technologies in user interfaces” 12© Brad Myers

Why is Brad Myers Qualified to Teach This? Worked in the MIT Architecture Machine Group (now Media lab) when SDMS was created Interned at Xerox PARC in the days of the Alto & Star ( ) “Incense” visualization Created one of the first commercial window managers at Three Rivers Computer – Perq (1980-4) PhD with Bill Buxton, on “Peridot” Created two influential toolkits in the 80’s and 90’s Garnet, Amulet Highly published at CHI and highly cited by UIST papers 13© Brad Myers

Why is this topic important? Because it’s ½ of the HCI = People + Computers picture Because technology is changing society rapidly, in big (and small) ways Because technology is where the rubber meets the road where your research can make the world a better place Because now is a critical time for technology & people 14© Brad Myers

Computing is Everywhere © Brad Myers15

Computing is Everywhere © Brad Myers16

Why is that? A small processor costs < $1 If you can add $2-$5 to the cost of something, you can add a processor if there is something of value to be gained (doesn’t have to be much) © Brad Myers17

Technology Side of HCI has Affected Everyone Our Interaction Techniques are used by everyone Scrolling, Copy-and-Paste, Window Managers, … All software is built with descendents of our tools UI Toolkits, Resource Editors, object-oriented programming, … We are researching the next set of interaction techniques and tools 18© Brad Myers

Why Study Interaction Techniques? Used extensively Everyone who uses a computer uses copy-paste, etc. So can have an enormous impact Interesting historically Why do we do things the way we do? Is there a good reason? Example: which way does the arrow point in a scroll bar? And new interaction techniques are created all the time: Patent on “Bounce at end of scrolling” for iPhone submitted by Bas Ording in 2007 (right before 1 st iPhone was released in 2007) US 7,469,381US 7,469,381 Try it! iPhone vs. Samsung “Pull down to refresh” – patent submitted in 2010 by Twitter, became popular in 2013! US 8,448,084 Many new CHI & UIST conference papers every year with new ones © Brad Myers19

Why Study Interaction Techniques, cont. Interaction Techniques have a high economic value Often the subject of patents and lawsuits Can’t patent overall look and feel “Apple Wins Over Jury in Samsung Patent Dispute, Awarded $1.05 Billion in Damages (Live Blog)” linklink “Jury orders Samsung to pay $290M to Apple in patent case” linklink Need new ones “Desktop metaphor” is getting tired Macs & PCs look and work pretty similar to each other and to the designs of the 1980’s (30 years ago) Text entry on smartphones is still a big barrier Selecting individual elements, characters on smartphones © Brad Myers20

Why Hard? Brad Myers 21© Brad Myers

Problem Appliances are too complex 22© Brad Myers

Problem Too many remotes 23© Brad Myers

Problem April 29, © Brad Myers

25 Good UIs on Successful Products Palm succeeded where other handhelds had failed due to a focus on usability: Fit into pocket Reliable gestural text input Commands immediately available Apple iPod lauded for design and user interface iTunes  entire service design Apple iPhone – unique UI Apple iPad – desirable Wii controller, vs. XBox, PS3 graphics & power © Brad Myers

Bad UIs can Sink Products & Companies Damage reputations Ford dropped in ratings due to touch screen interface “Despite Ford’s improvements in manufacturing quality, their overall ratings fell precipitously this year due solely to the poor software interaction on their dashboards.” – NYT, Cooper ReportNYTCooper Report “’annoying’ behavior of their driver-facing interactive systems that caused their ratings to plummet.” 26© Brad Myers

Nokia & RIM 27 © Brad Myers

Bad UIs Can Cause Disasters Aegis July 4, 1988; Iranian Airbus shootdown by the Vincennes Deaths in kids: “Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry (CPOE) System” Because it took so much longer, did not reduce errors overall Florida ballots (2000) 28© Brad Myers

Florida Ballots in © Brad Myers

Healthcare.gov Mostly were software engineering and project management problems, but site-decried-as-train-wreck/ site-decried-as-train-wreck/ website-is-crashing-because-backend-was-doomed-in-the-requirements- stage/ © Brad Myers30

Why are User Interfaces Difficult to Design? 31© Brad Myers

Why Hard to Design UIs? “It is easy to make things hard. It is hard to make things easy.” No silver bullet Seems easy, common sense, but seldom done right Once done right, however, seems “obvious” User Interface design is a creative process Designers have difficulty thinking like users Often need to understand task domain Can’t “unlearn” something 32© Brad Myers

Can’t Unlearn Something 33© Brad Myers

Why Difficult, 2 Specifications are always wrong: "Only slightly more than 30% of the code developed in application software development ever gets used as intended by end-users. The reason for this statistic may be a result of developers not understanding what their users need." -- Hugh Beyer and Karen Holtzblatt, "Contextual Design: A Customer-Centric Approach to Systems Design,“ ACM Interactions, Sep+Oct, 1997, iv.5, p. 62. Need for prototyping and iteration 34© Brad Myers

Why Difficult, 3 Tasks and domains are complex Word 1 (100 commands) vs. Word 2007 (>2000) MacDraw 1 vs. Illustrator BMW iDrive adjusts over 700 functions Existing theories and guidelines are not sufficient Too specific and/or too general Standard does not address all issues. Adding graphics can make worse Pretty  Easy to use Can ’ t just copy other designs Legal issues 35© Brad Myers

Why Difficult, 4 All UI design involves tradeoffs: Standards (style guides, related products) Graphic design (artistic) Technical writing (Documentation) Internationalization Performance Multiple platforms (hardware, browsers, etc.) High-level and low-level details External factors (social issues) Legal issues Time to develop and test (“time to market”) 36© Brad Myers

Why are Interaction Techniques Difficult to Design? 37© Brad Myers

“Interaction Techniques” Scroll bars, buttons, text fields But also: Drawing a new object in an editor Copy-and-paste Selecting a cell in a spreadsheet How high level? Text editor widget, but not Word Scroll bar is composed of buttons, etc. © Brad Myers38

More examples Visual Basic Physical controls 39© Brad Myers

Other names “Widgets” (Wikipedia: “GUI Widget”) Note that there are no cross references in Wikipedia between “Interaction Technique” and “Widget” See my video “All the Widgets”video But not the same as Apple dashboard widgetsdashboard widgets GUI “elements” “Gadgets” But not the same as Scott Hudson’s “Controls” (Windows) “Components” Too generic “Behaviors” = the interaction part © Brad Myers40

What is not an interaction technique? Whole applications (Microsoft Word) Dashboard “widgets” – small apps for the desktop Output only (no interactions) Visualizations But many come with specialized interactions, then they might count? Animations Movies … © Brad Myers41

Why are Interaction Techniques Hard to Design? Surprisingly large number of design decisions Individual differences and preferences Lots of details that impact human performance How far does the cursor move when you move the mouse 1 inch? Trick question – depends on mouse speed Complex formula developed through experimentation How far does the content move on an iPhone when you flick your finger? Needs to work for long distance, and highly accurate local movements Nokia phones released just after the iPhone got this all wrong © Brad Myers42

Example: check box How many “states” can it be in? © Brad Myers43

Example: check box How many “states” can it be in? Checked, not-checked, Disabled, not-disabled Hover, not-hover (can’t be hover+disabled) Pressed-inside, pressed-outside, not- pressed (can’t be pressed + disabled, can’t be pressed-inside + not-hover) Keyboard focus, not-focus 2^4 * 3 = 48, but many are not possible Often forget about the release-outside case & interface gets confused (Flash implementations) © Brad Myers44

Example2: Drawing a new object What happens when move upwards past start point? © Brad Myers45

Why are User Interfaces Difficult to Implement? 46© Brad Myers

What are the most difficult kinds of programs, in general? What properties make a task difficult to program? 47© Brad Myers

Why Are User Interfaces Hard to Implement? They are hard to design, requiring iterative implementation Not the waterfall model: specify, design, implement, test, deliver They are reactive and are programmed from the "inside-out" Event based programming More difficult to modularize They generally require multi-processing To deal with user typing; aborts Window refresh Window system as a different process Multiple input devices 48© Brad Myers

Why Hard to Implement? cont. There are real-time requirements for handling input events Output 60 times a second Keep up with mouse tracking Video, sound, multi-media Need for robustness No crashing, on any input Helpful error messages and recover gracefully Aborts Undo 49© Brad Myers

Why Hard to Implement? cont. Lower testability Few tools for regression testing Little language support Primitives in computer languages make bad user interfaces Enormous, complex libraries Features like object-oriented, constraints, multi-processing Complexity of the tools Full bookshelf for documentation of user interface frameworks MFC, Java Swing, VB.Net, etc. Difficulty of Modularization 50© Brad Myers

Examples Difference between displaying “hello” in console and displaying a blue rectangle in a window Difficulty to read a file name Readln() in Pascal, Java, C++, etc. Vs. tool in modern toolkits Complexity of the file dialog itself You must deal with aborting, undo, etc. 51© Brad Myers

Success of Tools Today’s tools are highly successful Window Managers, Toolkits, Interface Builders ubiquitous Most software built using them Are based on many years of HCI research Brad A. Myers. “A Brief History of Human Computer Interaction Technology.” ACM interactions. Vol. 5, no. 2, March, pp © Brad Myers