Presentation is loading. Please wait.

Presentation is loading. Please wait.

Interaction Devices Human Computer Interaction CIS 6930/4930 Section 4188/4186.

Similar presentations


Presentation on theme: "Interaction Devices Human Computer Interaction CIS 6930/4930 Section 4188/4186."— Presentation transcript:

1 Interaction Devices Human Computer Interaction CIS 6930/4930 Section 4188/4186

2 Interaction Performance 60s vs. Today 60s vs. Today Performance Performance Hz -> GHz Hz -> GHz Memory Memory k -> GB k -> GB Storage Storage k -> TB k -> TB Input Input punch cards -> punch cards -> Keyboards, Pens, tablets, mobile phones, mice, digital cameras, web cams Keyboards, Pens, tablets, mobile phones, mice, digital cameras, web cams Output Output 10 character/sec 10 character/sec Megapixel displays, color laser, surround sound, force feedback, VR Megapixel displays, color laser, surround sound, force feedback, VR Substantial bandwidth increase! Substantial bandwidth increase!

3 Interaction Performance Future? Future? Gestural input Gestural input Two-handed input Two-handed input 3D I/O 3D I/O Others: voice, wearable, whole body, eye trackers, data gloves, haptics, force feedback Others: voice, wearable, whole body, eye trackers, data gloves, haptics, force feedback Engineering research! Engineering research! Entire companies created around one single technology Entire companies created around one single technology Current trend: Current trend: Multimodal (using car navigation via buttons or voice) Multimodal (using car navigation via buttons or voice) Helps disabled (esp. those w/ different levels of disability) Helps disabled (esp. those w/ different levels of disability)

4 Keyboard and Keypads QWERTY keyboards been around for a long time QWERTY keyboards been around for a long time (1870s – Christopher Sholes) (1870s – Christopher Sholes) Cons: Not easy to learn Cons: Not easy to learn Pros: Familiarity Pros: Familiarity Stats: Stats: Beginners: 1 keystroke per sec Beginners: 1 keystroke per sec Average office worker: 5 keystrokes (50 wpm) Average office worker: 5 keystrokes (50 wpm) Experts: 15 keystrokes per sec (150 wpm) Experts: 15 keystrokes per sec (150 wpm) Is it possible to do better? Suggestions? Is it possible to do better? Suggestions?

5 Keyboard and Keypads Look at the piano for possible inspiration Look at the piano for possible inspiration Court reporter keyboards (one keypress = multiple letters or a word) Court reporter keyboards (one keypress = multiple letters or a word) 300 wpm, requires extensive training and use 300 wpm, requires extensive training and use Keyboard properties that matter Keyboard properties that matter Size Size large - imposing for novices, appears more complex large - imposing for novices, appears more complex mobile devices mobile devices Adjustable Adjustable Reduces RSI, better performance and comfort Reduces RSI, better performance and comfort Mobile phone keyboards, blackberry devices, etc. Mobile phone keyboards, blackberry devices, etc.

6 Keyboard Layouts QWERTY QWERTY Frequently used pairs far apart Frequently used pairs far apart Fewer typewriter jams Fewer typewriter jams Electronic approaches dont jam.. why use it? Electronic approaches dont jam.. why use it? DVOARK (1920s) DVOARK (1920s) 150 wpm->200 wpm 150 wpm->200 wpm Reducing errors Reducing errors Takes about one week to switch Takes about one week to switch Stops most from trying Stops most from trying ABCDE – style ABCDE – style Easier for non-typists Easier for non-typists Studies show no improvement vs. QWERTY Studies show no improvement vs. QWERTY Number pads Number pads Whats in the top row? Whats in the top row? Look at phones (slight faster), then look at calculators, keypads Look at phones (slight faster), then look at calculators, keypads Those for disabled Those for disabled Split keyboards Split keyboards KeyBowls orbiTouch (screenshot) KeyBowls orbiTouch (screenshot) Eyetrackers, mice Eyetrackers, mice Dasher - 2d motion with word prediction Dasher - 2d motion with word prediction

7 Keys Current keyboards have been extensively tested Current keyboards have been extensively tested Size Size Shape Shape Required force Required force Spacing Spacing Speed vs. error rates for majority of users Speed vs. error rates for majority of users Distinctive click gives audio feedback Distinctive click gives audio feedback Why membrane keyboards are slow (Atari 400?) Why membrane keyboards are slow (Atari 400?) Environment hazards might necessitate Environment hazards might necessitate Usually speed is not a factor Usually speed is not a factor

8 Keys Guidelines Special keys should be denoted Special keys should be denoted State keys (such as caps, etc.) should have easily noted states State keys (such as caps, etc.) should have easily noted states Special curves or dots for home keys for touch typists Special curves or dots for home keys for touch typists Inverted T Cursor movement keys are important (though cross is easier for novices) Inverted T Cursor movement keys are important (though cross is easier for novices) Auto-repeat feature Auto-repeat feature Improves performance, but only if repeat is customizable (motor impaired, young, old) Improves performance, but only if repeat is customizable (motor impaired, young, old) Two thinking points: Two thinking points: Why are home keys fastest to type? Why are home keys fastest to type? Why are certain keys larger? (Enter, Shift, Space bar) Why are certain keys larger? (Enter, Shift, Space bar) This is called Fitts Law This is called Fitts Law

9 Keypads for small devices PDAs, Cellphones, Game consoles PDAs, Cellphones, Game consoles Fold out keyboards Fold out keyboards Virtual keyboard Virtual keyboard Cloth keyboards (ElekSen) Cloth keyboards (ElekSen) Haptic feedback? Haptic feedback? Mobile phones Mobile phones Combine static keys with dynamic soft keys Combine static keys with dynamic soft keys Multi-tap a key to get to a character Multi-tap a key to get to a character Study: Predictive techniques greatly improve performance Study: Predictive techniques greatly improve performance Ex. LetterWise = 20 wpm vs 15 wpm multitap Ex. LetterWise = 20 wpm vs 15 wpm multitap Draw keyboard on screen and tap w/ pen Draw keyboard on screen and tap w/ pen Speed: 20 to 30 wpm (Sears 93) Speed: 20 to 30 wpm (Sears 93) Handwriting recognition (still hard) Handwriting recognition (still hard) Subset: Graffiti2 (uses unistrokes) Subset: Graffiti2 (uses unistrokes)

10 Pointing Devices Direct manipulation needs some pointing device Direct manipulation needs some pointing device Factors: Factors: Size of device Size of device Accuracy Accuracy Dimensionality Dimensionality Interaction Tasks: Interaction Tasks: Select – menu selection, from a list Select – menu selection, from a list Position – 1D, 2D, 3D (ex. paint) Position – 1D, 2D, 3D (ex. paint) Orientation – Control orientation or provide direct 3D orientation input Orientation – Control orientation or provide direct 3D orientation input Path – Multiple poses are recorded Path – Multiple poses are recorded ex. to draw a line ex. to draw a line Quantify – control widgets that affect variables Quantify – control widgets that affect variables Text – move text Text – move text Faster w/ less error than keyboard Faster w/ less error than keyboard Two types (Box 9.1) Two types (Box 9.1) Direct control – device is on the screen surface (touchscreen, stylus) Direct control – device is on the screen surface (touchscreen, stylus) Indirect control – mouse, trackball, joystick, touchpad Indirect control – mouse, trackball, joystick, touchpad

11 Direct-control pointing First device – lightpen First device – lightpen Point to a place on screen and press a button Point to a place on screen and press a button Pros: Pros: Easy to understand and use Easy to understand and use Very fast for some operations (e.g. drawing) Very fast for some operations (e.g. drawing) Cons: Cons: Hand gets tired fast! Hand gets tired fast! Hand and pen blocks view of screen Hand and pen blocks view of screen Fragile Fragile Evolved into the touchscreen Evolved into the touchscreen Pros: Very robust, no moving parts Pros: Very robust, no moving parts Cons: Depending on app, accuracy could be an issue Cons: Depending on app, accuracy could be an issue 1600x1600 res with acoustic wave 1600x1600 res with acoustic wave Must be careful about software design for selection (land-on strategy). Must be careful about software design for selection (land-on strategy). If you dont show a cursor of where you are selecting, users get confused If you dont show a cursor of where you are selecting, users get confused User confidence is improved with a good lift-off strategy User confidence is improved with a good lift-off strategy

12 Direct-control pointing Primarily for novice users or large user base Primarily for novice users or large user base Case study: Disney World Case study: Disney World Need to consider those who are: disabled, illiterate, hard of hearing, errors in usage (two touch points), etc. Need to consider those who are: disabled, illiterate, hard of hearing, errors in usage (two touch points), etc.

13 Indirect-Control Pointing Pros: Pros: Reduces hand-fatigue Reduces hand-fatigue Reduces obscuration problems Reduces obscuration problems Cons: Cons: Increases cognitive load Increases cognitive load Spatial ability comes more into play Spatial ability comes more into play Mouse Mouse Pros: Pros: Familiarity Familiarity Wide availability Wide availability Low cost Low cost Easy to use Easy to use Accurate Accurate Cons: Cons: Time to grab mouse Time to grab mouse Desk space Desk space Encumbrance (wire), dirt Encumbrance (wire), dirt Long motions arent easy or obvious (pick up and replace) Long motions arent easy or obvious (pick up and replace) Consider, weight, size, style, # of buttons, force feedback Consider, weight, size, style, # of buttons, force feedback

14 Indirect-Control Pointing Trackball Trackball Pros: Pros: Small physical footprint Small physical footprint Good for kiosks Good for kiosks Joystick Joystick Easy to use, lots of buttons Easy to use, lots of buttons Good for tracking (guide or follow an on screen object) Good for tracking (guide or follow an on screen object) Does it map well to your app? Does it map well to your app? Touchpoint Touchpoint Pressure-sensitive nubbin on laptops Pressure-sensitive nubbin on laptops Keep fingers on the home position Keep fingers on the home position

15 Indirect-Control Pointing Touchpad Touchpad Laptop mouse device Laptop mouse device Lack of moving parts, and low profile Lack of moving parts, and low profile Accuracy, esp. those w/ motor disabilities Accuracy, esp. those w/ motor disabilities Graphics Tablet Graphics Tablet Screen shot Screen shot comfort comfort good for cad, artists good for cad, artists Limited data entry Limited data entry

16 Comparing pointing devices Direct pointing Direct pointing Study: Faster but less accurate than indirect (Haller 84) Study: Faster but less accurate than indirect (Haller 84) Lots of studies confirm mouse is best for most tasks for speed and accuracy Lots of studies confirm mouse is best for most tasks for speed and accuracy Trackpoint < Trackballs & Touchpads < Mouse Trackpoint < Trackballs & Touchpads < Mouse Short distances – cursor keys are better Short distances – cursor keys are better Disabled prefer joysticks and trackballs Disabled prefer joysticks and trackballs If force application is a problem, then touch sensitive is preferred If force application is a problem, then touch sensitive is preferred Vision impaired have problems with most pointing devices Vision impaired have problems with most pointing devices Use multimodal approach or customizable cursors Use multimodal approach or customizable cursors Read Vanderheiden 04 for a case study Read Vanderheiden 04 for a case study Designers should smooth out trajectories Designers should smooth out trajectories Large targets reduce time and frustration Large targets reduce time and frustration

17 Example Five fastest places to click on for a right- handed user? Five fastest places to click on for a right- handed user?

18 Example What affects time? What affects time?

19 Fittss Law Paul Fitts (1954) developed a model of human hand movement Paul Fitts (1954) developed a model of human hand movement Used to predict time to point at an object Used to predict time to point at an object What are the factors to determine the time to point to an object? What are the factors to determine the time to point to an object? D – distance to target D – distance to target W – size of target W – size of target Just from your own experience, is this function linear? Just from your own experience, is this function linear? No, since if Target A is D distance and Target B is 2D distance, it doesnt take twice as long No, since if Target A is D distance and Target B is 2D distance, it doesnt take twice as long What about target size? Not linear there either What about target size? Not linear there either MT = a + b log 2 (D/W + 1) MT = a + b log 2 (D/W + 1) a = time to start/stop in seconds (empirically measured per device) a = time to start/stop in seconds (empirically measured per device) b = inherent speed of the device (empirically measured per device) b = inherent speed of the device (empirically measured per device) Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm Ans: log 2 (14/2 + 1) = 900 ms Ans: log 2 (14/2 + 1) = 900 ms Really a slope-intercept model Really a slope-intercept model

20 Fittss Law MT = a + b log 2 (D/W + 1) MT = a + b log 2 (D/W + 1) a = time to start/stop in seconds (empirically measured per device) a = time to start/stop in seconds (empirically measured per device) b = inherent speed of the device (empirically measured per device) b = inherent speed of the device (empirically measured per device) Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm Ans: log 2 (14/2 + 1) = 900 ms Ans: log 2 (14/2 + 1) = 900 ms Question: If I wanted to half the pointing time (on average), how much do I change the size? Question: If I wanted to half the pointing time (on average), how much do I change the size? Proven to provide good timings for most age groups Proven to provide good timings for most age groups Newer versions taken into account Newer versions taken into account Direction (we are faster horizontally than vertically) Direction (we are faster horizontally than vertically) Device weight Device weight Target shape Target shape Arm position (resting or midair) Arm position (resting or midair) 2D and 3D (Zhai 96) 2D and 3D (Zhai 96)

21 Very Successfully Studied Applies to Applies to Feet, eye gaze, head mounted sights Feet, eye gaze, head mounted sights Many types of input devices Many types of input devices Physical environments (underwater!) Physical environments (underwater!) User populations (even retarded and drugged) User populations (even retarded and drugged) Drag & Drop and Point & Click Drag & Drop and Point & Click Limitations Limitations Dimensionality Dimensionality Software accelerated pointer motion Software accelerated pointer motion Training Training Trajectory Tasks (Accot-Zhai Steering Law) Trajectory Tasks (Accot-Zhai Steering Law) Decision Making (Hicks Law) Decision Making (Hicks Law) Results (what does it say about) Results (what does it say about) Buttons and widget size? Buttons and widget size? Edges? Edges? Popup vs. pull-down menus Popup vs. pull-down menus Pie vs. Linear menus Pie vs. Linear menus iPhone/web pages (real borders) vs. monitor+mouse (virtual borders) iPhone/web pages (real borders) vs. monitor+mouse (virtual borders) Interesting readings: Interesting readings:

22 Precision Pointing Movement Time Study: Sears and Shneiderman 91 Study: Sears and Shneiderman 91 Broke down task into gross and fine components for small targets Broke down task into gross and fine components for small targets PPMT = a + b log 2 (D/W+1) + c log 2 (d/W) PPMT = a + b log 2 (D/W+1) + c log 2 (d/W) c – speed for short distance movement c – speed for short distance movement d – minor distance d – minor distance Notice how the overall time changes with a smaller target. Notice how the overall time changes with a smaller target. Other factors Other factors Age (Pg. 369) Age (Pg. 369) Research: How can we design devices that produce smaller constants for the predictive equation Research: How can we design devices that produce smaller constants for the predictive equation Two handed Two handed Zooming Zooming

23 Novel Devices Themes: Themes: Make device more diverse Make device more diverse Users Users Task Task Improve match between task and device Improve match between task and device Improve affordance Improve affordance Refine input Refine input Feedback strategies Feedback strategies Foot controls Foot controls Already used in music where hands might be busy Already used in music where hands might be busy Cars Cars Foot mouse was twice as slow as hand mouse Foot mouse was twice as slow as hand mouse Could specify modes Could specify modes

24 Novel Devices Eye-tracking Eye-tracking Accuracy 1-2 degrees Accuracy 1-2 degrees selections are by constant stare for ms selections are by constant stare for ms How do you distinguish w/ a selection and a gaze? How do you distinguish w/ a selection and a gaze? Combine w/ manual input Combine w/ manual input Multiple degree of freedom devices Multiple degree of freedom devices Logitech Spaceball and SpaceMouse Logitech Spaceball and SpaceMouse Ascension Bird Ascension Bird Polhemus Liberty and IsoTrack Polhemus Liberty and IsoTrack

25 Novel Devices Boom Chameleon Boom Chameleon Pros: Natural, good spatial understanding Pros: Natural, good spatial understanding Cons: limited applications, hard to interact (very passive) Cons: limited applications, hard to interact (very passive) DataGlove DataGlove Pinch glove Pinch glove Gesture recognition Gesture recognition American Sign Language, musical director American Sign Language, musical director Pros: Natural Pros: Natural Cons: Size, hygiene, accuracy, durability Cons: Size, hygiene, accuracy, durability

26 Novel Devices Haptic Feedback Haptic Feedback Why is resistance useful? Why is resistance useful? SensAble Technologys Phantom SensAble Technologys Phantom Cons: limited applications Cons: limited applications Sound and vibration are easier and can be a good approximation Sound and vibration are easier and can be a good approximation Rumble pack Rumble pack Two-Handed input Two-Handed input Different hands have different precision Different hands have different precision Non-dominant hand selects fill, the other selects objects Non-dominant hand selects fill, the other selects objects Ubiquitous Computing and Tangible User Interface Ubiquitous Computing and Tangible User Interface Active Badges allows you to move about the house w/ your profile Active Badges allows you to move about the house w/ your profile Which sensors could you use? Which sensors could you use? Elderly, disabled Elderly, disabled Research: Smart House Research: Smart House Myron Kruger – novel user participation in art (Lots of exhibit art at siggraph) Myron Kruger – novel user participation in art (Lots of exhibit art at siggraph)

27 Novel Devices Paper/Whiteboards Paper/Whiteboards Video capture of annotations Video capture of annotations Record notes (special tracked pens Logitech digital pen) Record notes (special tracked pens Logitech digital pen) Handheld Devices Handheld Devices PDA PDA Universal remote Universal remote Help disabled Help disabled Read LCD screens Read LCD screens Rooms in building Rooms in building Maps Maps Interesting body-context-sensitive. Interesting body-context-sensitive. Ex. hold PDA by ear = phone call answer. Ex. hold PDA by ear = phone call answer.

28 Novel Devices Miscellaneous Miscellaneous Shapetape – reports 3D shape. Shapetape – reports 3D shape. Tracks limbs Tracks limbs Engineer for specific app (like a gun trigger connected to serial port) Engineer for specific app (like a gun trigger connected to serial port) Pros: good affordance Pros: good affordance Cons: Limited general use, time Cons: Limited general use, time

29 Speech and Auditory Interfaces Theres the dream Theres the dream Then theres reality Then theres reality Practical apps dont really require freeform discussions with a computer Practical apps dont really require freeform discussions with a computer Goals: Goals: Low cognitive load Low cognitive load Low error rates Low error rates Smaller goals: Smaller goals: Speech Store and Forward (voice mail) Speech Store and Forward (voice mail) Speech Generation Speech Generation Currently not too bad, low cost, available Currently not too bad, low cost, available

30 Speech and Auditory Interfaces Bandwidth is much lower than visual displays Bandwidth is much lower than visual displays Ephemeral nature of speech (tone, etc.) Ephemeral nature of speech (tone, etc.) Difficulty in parsing/searching (Box 9.2) Difficulty in parsing/searching (Box 9.2) Types Types Discrete-word recognition Discrete-word recognition Continuous speech Continuous speech Voice information Voice information Speech generation Speech generation Non-speech auditory Non-speech auditory If you want to do research here, lots of research in the audio, audio psychology, and DSP field you should understand If you want to do research here, lots of research in the audio, audio psychology, and DSP field you should understand

31 Discrete-Word Recognition Individual words spoken by a specific person Individual words spoken by a specific person Command and control Command and control 90-98% for word vocabularies 90-98% for word vocabularies Training Training Speaker speaks the vocabulary Speaker speaks the vocabulary Speaker-independent Speaker-independent Still requires Still requires Low noise operating environment Low noise operating environment Microphones Microphones Vocabulary choice Vocabulary choice Clear voice (language disabled are hampered, stressed) Clear voice (language disabled are hampered, stressed) Reduce most questions to very distinct answers (yes/no) Reduce most questions to very distinct answers (yes/no)

32 Discrete-Word Recognition Helps: Helps: Disabled Disabled Elderly Elderly Cognitive challenged Cognitive challenged User is visually distracted User is visually distracted Mobility or space restrictions Mobility or space restrictions Apps: Apps: Telephone-based info Telephone-based info Study: much slower for cursor movement than mouse or keyboard (Christian 00) Study: much slower for cursor movement than mouse or keyboard (Christian 00) Study: choosing actions (such as drawing actions) improved performance by 21% (Pausch 91) and word processing (Karl 93) Study: choosing actions (such as drawing actions) improved performance by 21% (Pausch 91) and word processing (Karl 93) However acoustic memory requires high cognitive load (> than hand/eye) However acoustic memory requires high cognitive load (> than hand/eye) Toys are successful (dolls, robots). Accuracy isnt as important Toys are successful (dolls, robots). Accuracy isnt as important Feedback is difficult Feedback is difficult

33 Continuous Speech Recognition Dictation Dictation Error rates and error repair are still poor Error rates and error repair are still poor Higher cognitive load, could lower overall quality Higher cognitive load, could lower overall quality Why is it hard? Why is it hard? Recognize boundaries (normal speech blurs them) Recognize boundaries (normal speech blurs them) Context sensitivity Context sensitivity How to wreck a nice beach How to wreck a nice beach Much training Much training Specialized vocabularies (like medical or legal) Specialized vocabularies (like medical or legal) Apps: Apps: Dictate reports, notes, letters Dictate reports, notes, letters Communication skills practice (virtual patient) Communication skills practice (virtual patient) Automatic retrieval/transcription of audio content (like radio, CC) Automatic retrieval/transcription of audio content (like radio, CC) Security/user ID Security/user ID

34 Voice Information Systems Use human voice as a source of info Use human voice as a source of info Apps: Apps: Tourist info Tourist info Museum audio tours Museum audio tours Voice menus (Interactive Voice Response IVR systems) Voice menus (Interactive Voice Response IVR systems) Use speech recognition to also cut through menus Use speech recognition to also cut through menus If menus are too long, users get frustrated If menus are too long, users get frustrated Cheaper than hiring 24 hr/day reps Cheaper than hiring 24 hr/day reps Voice mail systems Voice mail systems Interface isnt the best Interface isnt the best Get in your car Get in your car Also helps with non-tech savvy like the elderly Also helps with non-tech savvy like the elderly Potentially aides with Potentially aides with Learning (engage more senses) Learning (engage more senses) Cognitive load (hypothesize each sense has a limited bandwidth) Cognitive load (hypothesize each sense has a limited bandwidth) Think ER, or fighter jets Think ER, or fighter jets

35 Speech Generation Play back speech (games) Play back speech (games) Combine text (navigation systems) Combine text (navigation systems) Careful evaluation! Careful evaluation! Speech isnt always great Speech isnt always great Door is ajar – now just a tone Door is ajar – now just a tone Use flash Use flash Supermarket scanners Supermarket scanners Often times a simple tone is better Often times a simple tone is better Why? Cognitive load Why? Cognitive load Thus cockpits and control rooms need speech Thus cockpits and control rooms need speech Competes w/ human-human communication Competes w/ human-human communication

36 Speech Generation Ex: Text-to-Speech (TTS) Ex: Text-to-Speech (TTS) Latest TTS uses multiple syllabi to make generated speech sound better Latest TTS uses multiple syllabi to make generated speech sound better Robotic speech could be desirable to get attention Robotic speech could be desirable to get attention All depends on app All depends on app Thus dont assume one way is the best, you should user test Thus dont assume one way is the best, you should user test Apps: TTS for blind, JAWS Apps: TTS for blind, JAWS Web-based voice apps: VoiceXML and SALT (tagged web pages). Web-based voice apps: VoiceXML and SALT (tagged web pages). Good for disabled, and also for mobile devices Good for disabled, and also for mobile devices Use if Use if Message is short Message is short Requires dynamic responses Requires dynamic responses Events in time Events in time Good when visual displays arent that useful. When? Good when visual displays arent that useful. When? Bad lighting, vibrations (say liftoff) Bad lighting, vibrations (say liftoff)

37 Non-speech Auditory Interface Audio tones that provide information Audio tones that provide information Major Research Area Major Research Area Sonification – converting information into audio Sonification – converting information into audio Audiolization Audiolization Auditory Interfaces Auditory Interfaces Browsers produced a click when you clicked on a link Browsers produced a click when you clicked on a link Increases confidence Increases confidence Can do tasks without visual cognitive load Can do tasks without visual cognitive load Helps figure out when things are wrong Helps figure out when things are wrong Greatly helps visually impaired Greatly helps visually impaired

38 Non-speech Auditory Interface Terms: Terms: Auditory icons – familiar sounds (record real world sound and play it in your app) Auditory icons – familiar sounds (record real world sound and play it in your app) Earcons – new learned sounds (door ajar) Earcons – new learned sounds (door ajar) Role in video games is huge Role in video games is huge Emotions, Tension, set mood Emotions, Tension, set mood To create 3D sound To create 3D sound Need to do more than stereo Need to do more than stereo Take into account Head-related transfer function (HRTF) Take into account Head-related transfer function (HRTF) Ear and head shape Ear and head shape New musical instruments New musical instruments Theremin Theremin New ways to arrange music New ways to arrange music

39 Displays Primary Source of feedback Primary Source of feedback Properties: Properties: Physical Dimension Physical Dimension Resolution Resolution Color Depth and correctness Color Depth and correctness Brightness, contrast, glare Brightness, contrast, glare Power Power Refresh rate Refresh rate Cost Cost Reliability Reliability # of users # of users

40 Display Technology Monochrome displays (single color) Monochrome displays (single color) Low cost Low cost Greater intensity range (medical) Greater intensity range (medical) Color Color Raster Scan CRT Raster Scan CRT LCD – thin, bright LCD – thin, bright Plasma – very bright, thin Plasma – very bright, thin LED – large public displays LED – large public displays Electronic Ink – new product w/ tiny capsules of negative black particles and positive white Electronic Ink – new product w/ tiny capsules of negative black particles and positive white Braille – refreshable cells with dots that rise up Braille – refreshable cells with dots that rise up

41 Large Displays Wall displays Wall displays Informational Informational Control rooms, military, flight control rooms, emergency response Control rooms, military, flight control rooms, emergency response Provides Provides System overview System overview Increases situational awareness Increases situational awareness Effective team review Effective team review Old: Array of CRTs Old: Array of CRTs Interactive Interactive Require new interaction methods (freehand sketch, PDAs) Require new interaction methods (freehand sketch, PDAs) Local and remote collaboration Local and remote collaboration Art, engineering Art, engineering

42 Large Displays Multiple Desktop Displays Multiple Desktop Displays Multiple CRTs or Flat panels for large desktops Multiple CRTs or Flat panels for large desktops Cheap Cheap Familiar Familiar Spatial divide up tasks Spatial divide up tasks Comparison tasks are easier Comparison tasks are easier Too much info? Too much info? HMD HMD Eventually -> Every surface a pixel Eventually -> Every surface a pixel

43 Mobile device displays Applications Applications Personal Personal Reprogrammable picture frames Reprogrammable picture frames Digital family portrait (GaTech) Digital family portrait (GaTech) Business Business PDAs, cellphones PDAs, cellphones Medical Medical Monitor patients Monitor patients Research: Modality Translation Services (Trace Center – University of Wisconsin) Research: Modality Translation Services (Trace Center – University of Wisconsin) As you move about it auto converts data, info, etc. for you As you move about it auto converts data, info, etc. for you

44 Mobile device displays Actions on mobile devices Actions on mobile devices Monitor information and alert (calendar) Monitor information and alert (calendar) Gather then spread out information (phone) Gather then spread out information (phone) Participate in groups and relate to individual (networked devices) Participate in groups and relate to individual (networked devices) Locate services and identify objects (GPS car system) Locate services and identify objects (GPS car system) Capture and then share info (phone) Capture and then share info (phone)

45 Mobile device displays Guidelines for design Guidelines for design Bergman 00, Weiss, 02 Bergman 00, Weiss, 02 Industry led research and design case studies (Lindholm 03) Industry led research and design case studies (Lindholm 03) Typically short in time usage (except handheld games) Typically short in time usage (except handheld games) Optimize for repetitive tasks (rank functions by frequency) Optimize for repetitive tasks (rank functions by frequency) Research: new ways to organize large amounts of info on a small screen Research: new ways to organize large amounts of info on a small screen Study: Rapid Serial Visual Presentation (RSVP) presents text at a constant speed (33% improvement Oquist 03) Study: Rapid Serial Visual Presentation (RSVP) presents text at a constant speed (33% improvement Oquist 03) Searching and web browsing still very poor performance Searching and web browsing still very poor performance Promising: Hierarchical representation (show full document and allow user to select where to zoom into) Promising: Hierarchical representation (show full document and allow user to select where to zoom into)

46 Animation, Image, and Video Content quality has also greatly increased Content quality has also greatly increased 3D rendering is near life-like 3D rendering is near life-like Digital Photography is common Digital Photography is common Scanned documents Scanned documents Video compression Video compression Multimedia considerations for the disabled Multimedia considerations for the disabled Printers Printers 3D Printers create custom objects from 3D models 3D Printers create custom objects from 3D models


Download ppt "Interaction Devices Human Computer Interaction CIS 6930/4930 Section 4188/4186."

Similar presentations


Ads by Google