Download presentation
Presentation is loading. Please wait.
1
Direct Manipulation Interfaces
Why buy and use a product?
2
Direct Manipulation Interfaces
Positive acceptance of an application Mastery of the interface Competence in performing tasks Ease in learning originally and in assimilating advanced features Confidence in the capacity to retain mastery over time Enjoyment in using the interface Eagerness to show off interfaces to novices Desire to explore more powerful aspects Welcome to Lesson 8 Part A – Direct Manipulation Interfaces In this unit we will explore the evolution of user interface design including: Command – Line Interfaces Video Games Computer Aided Design Applications 3-d Interfaces Teleoperation Applications Telemedicine Virtual Reality Interfaces Augmented Reality Interfaces Let’s first start with the factors that pertain to the user having a positive reaction to an application: First - Mastery of the interface refers to the fact that users tend to accept an application the more they invest time into mastering the application Another factor related to acceptance is whether feels Competent in performing their task Ease in learning and in assimilating advanced features into the user’s knowledge of an application are also important factors Additionally, the more Confidence in their ability to retain mastery over time is also a key factor Acceptance is also related to Enjoyment in using the interface, especially for applications where use is discretionary There is also and ego factor in that acceptance increases as the user becomes more Eager to show off their knowledge of the interfaces to novices Finally, a users Desire to explore more powerful aspects of an application also is an indication of acceptance
3
Direct Manipulation Interfaces
Attributes of direct-manipulation interfaces Visibility of the objects and actions of interest Example: Driving an automobile Rapid, reversible, incremental actions Replacement of types commands by a pointing action on the object of interest Example: Dragging a file to a trash can versus “rm file.doc” What about: “rm file*.doc”? What about “rm *.*” with no undo? Now let’s look at direct manipulation interfaces and keep in the back of our minds the acceptance of various types of applications and how they pertain to direct manipulation style. Attributes of direct-manipulation interfaces include making the objects and actions related to interface objects visible to the user For Example when operating an automobile the user needs an unobstructed view of the road, and the instrument panel including speed and other indicators. Direct manipulation interfaces support Rapid, reversible, and incremental actions (e.g., undo) Direct manipulation interfaces enable the user to point at an object to perform an action as opposed to typing commands. For Example if the goal of the user is to erase a file, they can just: Drag a file to a trash can icon If a command line interface is used then the user would need to type a command of “rm file.doc” As such, the user needs to learn the language of the application to perform functions related to their task The language builds in that a command like: “rm file*.doc”? Would delete every file starting with the first four characters of file. The star is a wildcard. Taken to an extreme case, “rm *.*” would delete all files on your computer with no warning and no undo. In this case the application could become very unfriendly to the user
4
Direct Manipulation Interfaces
Extensions of direct manipulation Virtual reality – users are in an immersed environment Reality is blocked out via a head-mounted display Hand gestures (via a data glove) allow users to point, select, grasp and navigate Augmented reality – user remains in normal surroundings, but adds a transparent overlay. Examples – labeled buildings, hidden plumbing Google: Project Glass Tangible user interfaces – users manipulate physical objects Example – putting several plastic blocks near each other to create an office floor plan Extensions of direct manipulation include Virtual reality – where users are in an immersed in a virtual environment Reality is blocked out via a head-mounted display and Hand gestures (via a data glove) allow users to point, select, grasp and navigate objects in the virtual world In the case of Augmented reality, users remains in normal surroundings, but the application adds a transparent overlay on the physical world An Example includes labeled buildings. Specifically, the user can move around a city and the names and addresses of buildings can be overlaid on the buildings (via special glasses worn by the user) For Tangible user interfaces users manipulate actual physical objects that map to a computer display For Example – putting several plastic blocks near each other to create an office floor plan
5
Direct Manipulation Interfaces
6
Direct Manipulation Interfaces
Command-line versus display editors versus word processors The Tubeless Interface The tubeless interface refers to a time when computer monitors did not exist. People used keypunch machines that stamped holes in computer cards. The cards were placed in a card reader and a computer program was executed as a result of the cards being read. These applications were largely mathematical, calculating complex equations. However, believe it or not, there were used for word processing. In fact, I actually wrote my senior thesis using computer cards.
7
Direct Manipulation Interfaces
Command-line versus display editors versus word processors Single-line and Multi-line Editors (e.g., IBM MVS, VM, TSO, JCL) Line printers were then developed where the user did not need to type cards, but could directly type onto a keyboard, without the monitor. The user would type lines as see in the diagram. Even if the user was writing a paper, they needed to know the job control language Need to tell the main frame computer how much memory to allocate to the job and job priority (among other items).
8
Direct Manipulation Interfaces
Command-line versus display editors versus word processors Single-line and Multi-line Editors versus WYSIWYG (what you see is what you get) editors (e.g., Microsoft Word – early 1990s) Example: Three basic modes of vi Command mode (Telling the computer what to do: Low level commands, e.g., move the cursor to the right one character) Default when you enter vi. Most letters, or short sequences of letters, that you type will be interpreted as commands Pressing Esc when you're in command mode, your terminal will beep at you. This is a very good way to tell when you're in command mode Insert mode (Entering the content) Whatever you type is inserted in the file at the cursor position Press Esc to end insert mode, and return to command mode. Line mode (Telling the computer what to: High level commands, e.g. Save) To enter line mode from command mode, type a colon ( : ) Your cursor moves to the bottom of the screen, by a colon prompt. Type a line mode command, then press Enter. Now let’s contrast Command-line versus display editors where the task is word processing (or simply, writing a paper) Single-line and Multi-line Editors versus WYSIWYG (what you see is what you get) editors (e.g., Microsoft Word – early 1990s) Let’s start with the Example of a Unix based word processing application that was used in the 1960s and 1970s. The name of the application was vi: Vi had Three basic modes: Command mode (Telling the computer what to do: Low level commands, e.g., move the cursor to the right one character) When you first started the vi application you were in command mode, that is the computer was waiting for the user to tell it to do something. As such, Most letters, or short sequences of letters, that you type were interpreted as commands Pressing Esc when you're in command mode, resulted in your terminal will beeping at you. This is how you could tell when your were in command mode The second mode was Insert (this is the mode you are in when you Enter the content of your document) Whatever was typed was inserted in the file at the cursor position Pressing Esc to ended insert mode, and returned the user to command mode. The third mode was Line mode (Telling the computer what to: High level commands, such as Save) To enter line mode from command mode, the user would type a colon ( : ) Your cursor moves to the bottom of the screen, by a colon prompt Where the command would be entered by the user. The user would Type a line mode command, then press Enter. So at this point, the goal of creating a document is easier than just typing keypunch cards, but is still a long way from the word processing tools of today.
9
Direct Manipulation Interfaces
Command-line versus display editors versus word processors Single-line and Multi-line Editors (e.g., vi) Starting vi and Saving Files Starting vi: vi filename (start editing filename, create it if necessary) Saving the file you're working on and/or leaving vi: :wq (write the file to disk and quit) Quit without saving any changes: :q! :w! newfile (write all lines from the entire current file into the file 'newfile', overwriting any existing newfile) :n,m w! newfile (write the lines from n to m, inclusive, into the file newfile, overwriting any existing newfile) At the command prompt, the user would type “vi filename” (the user would then start editing the file. This is similar to double clicking on a icon today. To Save a file you're working on and exit the application, the user would type “:wq” (write the file to disk and quit) Quitting without saving any changes: was “:q!”. This is similar to selecting “Exit” from the “File” menu in Microsoft word.
10
Direct Manipulation Interfaces
Command-line versus display editors versus word processors Single-line and Multi-line Editors Type To Move To h one space to the left (also try left arrow) j one line down (also try down arrow) k one line up (also try up arrow) l one space to the right (also try right arrow) $ end of current line ^ beginning of current line Enter beginning first word on the next line G end of file :n line n; use :0 to move the beginning of the file w beginning of next word; 5w moves to the beginning of the 5th word to the right e end of next word b beginning of previous word Ctrl-b one page up Ctrl-f one page down % the matching (, ), [, ], {, or } (Press % with your cursor on one of these characters to move your cursor its mate.) To gain an appreciation for how advanced word processing software has become, scan over the following list of commands. Remember, that vi required that the user enter the command mode to control the cursor. There was no mouse, point and click type interface. Many of the commands just involved moving the cursor to a location in the document.
11
Direct Manipulation Interfaces
Command-line versus display editors versus word processors Single-line and Multi-line Editors Searching for text Inserting text Type To /string search down for string ?string search up for string n repeat last search from present position Type To a append starting right of cursor A append at the end of the current line i insert starting left of cursor I insert at beginning of the current line o open line below cursor, then enter insert mode O open line above cursor, then enter insert mode :r newfile add the contents of the file newfile starting below the current line In this example, the user needed to enter the command mode just to search for text, or insert text into a document. Again, no mouse positioning.
12
Direct Manipulation Interfaces
Command-line versus display editors versus word processors Single-line and Multi-line Editors Deleting text Type To x delete single character; 5x deletes 5 characters dw delete word; 5dw deletes 5 words dd delete line; 5dd deletes 5 rows cw delete word, leaves you in insert mode (i.e. change word) cc change line -- delete line and start insert mode s change character -- delete character and start insert mode D delete from cursor to end of line C change from cursor to end of line -- delete and start insert mode u undo last change U undo all changes to current line J join current line with line that follows (press Enter in insert mode to split line) Where are commands needed just to delete text.
13
Direct Manipulation Interfaces
Command-line versus display editors versus word processors Single-line and Multi-line Editors Cutting and Pasting Miscellaneous Commands Type To xp transpose two characters (two commands, x followed by p) yy yank (i.e. copy) one line into a general buffer (5yy to yank 5 lines) "ayy yank into the buffer named a P put the general buffer back before the current line "aP put from buffer a before current line p put the general buffer back after the current line "ap put from buffer a after the current line And here are commands just for cutting an pasting text. A huge number of documents were created using vi. The user needed to learn an entire language just to create a document. Now the question. How well was the vi application accepted. The answer, it was greatly accepted by the engineering and academic sectors. However, not so much by the public sector. It should be noted though, that applications such as vi served as the foundation for word processing applications that are in existence today. Type To Ctrl-g show line number of current line Ctrl-l redraw the entire display
14
Direct Manipulation Interfaces
Command-line versus display editors versus word processors Early 1980s – Text editing was done with line-oriented command languages Nroff/troff: Unix based word processor .nf = no-fill, you use for graphs or text that you don't want spaces to be ignored .ce # = centers by the # of sentences you enter .ti # = .ti indents but only for one line, so if you have one sentence that needs to be indented 8 spaces but the rest of the page is indented 3 you can use .ti 8 for that single sentence So now that the vi user of years ago could open files, save files, move the cursor and enter text, how did a document get created? The answer was that applications were build where the user could basically program a document. Examples of such applications included Nroff and Troff. The user would enter a command (e.g., .ce) which told the computer program that when the document was printed to center the text on the page. The input provided by the user is to the left, the output is on the right. Again, what progress has been made in the word processing software of today where the document is printed in the same form as entered by the user. No commands necessary. .fi = fill-in, extra space will be ignored and text that is entered like this, the fill-in command will continue until you enter a .nf command and vice-versa .in # = # is the number of spaces you want the text indented, this command will be in place until you put in a new .in command...if you want something to not be indented you put in .in 0
15
Direct Manipulation Interfaces
Command-line editors Who would use these interfaces? Why? So who would use vi, troff and nroff, and why would they use them?
16
Direct Manipulation Interfaces
Command-line editors Who would use these interfaces? Why? Mastery of the interface Competence in performing tasks Ease in learning originally and in assimilating advanced features Confidence in the capacity to retain mastery over time Enjoyment in using the interface Eagerness to show off interfaces to novices Desire to explore more powerful aspects Simply because they were highly accepted by the users. They felt the Mastery of the interface They felt Competent in performing tasks their tasks Even though the applications were difficult to learn, there was a language structure which evolved the end user to assimilating advanced features Users developed a Confidence in the capacity to retain mastery over time And using the applications actually became enjoyable Of course, with mastery came an Eagerness for users to show off their knowledge to novices The user group also shared the personality attribute of a Desire to explore more powerful aspects of the application
17
Direct Manipulation Interfaces
Command-line editors Who would use these interfaces? Why? BETTER THAN A TYPEWRITER BETTER THAN A SLIDE RULE BETTER THAN A CALCULATOR In the final analysis, the computer applications surpassed the existing technologies of the day. The computer processing application vi was better than a typewriter because revisions did not require retyping the entire paper. A calculator was much easier to use than a slide rule.
18
Direct Manipulation Interfaces
Advantages of WYSIWYG Editors Users see a full page of text 20 to 60 lines provides a context for each sentence The document is seen as it will appear when printed Eliminating the clutter of formatting commands Cursor action is visible Indicates where to focus attention and apply action Cursor motion is natural Arrow keys or mouse provide natural physical mechanisms for moving the cursor Labeled icons make frequent actions rapid Toolbar for frequent actions Immediate display of the results of an action Example: Clicking a button to center text provides immediate result Rapid response and display Full page of text in a fraction of a second Easily reversible actions Example: Undo, backspace Welcome to Lesson 8 Part B What you see is what you get word processing editors introduced the following advantages: First Users see a full page of text The document is seen as it will appear when printed WYSIWYG Eliminates the clutter of formatting commands Cursor action is visible – which Indicates where to focus attention and apply action Cursor motion is natural – the use of Arrow keys or the mouse provide natural physical mechanisms for moving the cursor Labeled icons make frequent actions rapid WYSIWYG provides Immediate display of the results of an action (e.g., Clicking a button to center text) The editors provides Rapid response and display of Full page of text in a fraction of a second WYSIWYG provides Easily reversible actions Example: Undo, backspace
19
Direct Manipulation Interfaces
Technology advancements evolving from word processing Integration of graphics, spreadsheets, animations, photographs, etc. Desktop publishing software Presentation software Hypermedia environments and the World Wide Web (hyperlinks to documents) Improved macro facilities (e.g., construct, save and edit sequences of frequently used actions) Spell checkers and thesauri Grammar checkers Use of passive voice Excessive use of certain words Lack of parallel construction Document assemblers Contracts Wills Now that we have looked at the evolution of word processing applications, let’s look at some additional enhancements That have been added to such applications to enhance their utility: First, there is east Integration of graphics, spreadsheets, animations, and photographs, directly into word processing documents Desktop publishing software has grow as an industry directly as a result of such enhancements Presentation software has become vital to the communication of information spanning the corporate and academic sectors Hypermedia environments and the World Wide Web (hyperlinks to documents) have been added such that word processing documents can easily be linked to related information. Improved macro facilities (e.g., construct, save and edit sequences of frequently used actions) In addition, Spell checkers and thesauri have been added to word processing applications Grammar checkers which check for items such as User of passive voice Excessive use of certain words and Lack of parallel construction Document assemblers have also come into existence where the end user can perform tasks that previously required the assistance of a professional (e.g., an attorney) Examples include Contracts and Wills
20
Direct Manipulation Interfaces
Spreadsheets 1979 – VisiCalc from a Harvard Business School student 254 rows and 63 columns Functions within a cell as it relates to other cells Simulation of an accountants spreadsheet Lotus dominated the market in the 1980s Today Excel dominates Graphics displays Multiple windows Statistical routines Database access (e.g., Price List to Service Catalog) Spreadsheet based applications have also come into existence and have had a significant impact on the business and accounting sectors. In 1979 – VisiCalc was developed by a Harvard Business School student The application as PC based and consistent of 254 rows and 63 columns It basically Simulated \ an accountants spreadsheet And Functions specified within a cell could be related to other cells (e.g., sum a column of values) By the 1980s Lotus dominated the market for spreadsheet software Today Microsoft Excel dominates, and provide capabilites such as Graphics displays Multiple windows Statistical routines Database access (e.g., Price List to Service Catalog)
21
Direct Manipulation Interfaces
Spatial Data Management Direct Touch Displays – Nicholas Negroponte at MIT Spatial Data Management refers to applications that provide the capabilities of having the user navigate through space Years ago Nicholas Negroponte at MIT developed an application which Display a map of the world and able to zoom in on the Pacific Ocean to see markers for convoys of military ships. The application involved Joystick input where the user could adjust the view of the application along X and Y coordinates. Today Google Earth is a powerful application that is available over the internet. So what started as a concept and limited application years ago has evolved into An application for the masses. Check out the Google Earth video to see their latest capabilities. ArcView – ESRI is an application that combines geographic information with crime related information with the goal of improving law enforcement It supports various views so the user can locate locations related to crime types (e.g., burglaries, assaults, and threfts) The user can also view crime data by location and time (e.g., time of day, day of week, or month) The data can also be used to identify location relative to specific crimes (e.g., high car theft areas) The user can also Select type of information to display (roads, population, rainfall, topography, and political boundaries)
22
Direct Manipulation Interfaces
Spatial Data Management ArcView – ESRI Global Information Systems (Demo) Select type of information to display (roads, population, rainfall, topography, political boundaries Spatial Data Management refers to applications that provide the capabilities of having the user navigate through space Years ago Nicholas Negroponte at MIT developed an application which Display a map of the world and able to zoom in on the Pacific Ocean to see markers for convoys of military ships. The application involved Joystick input where the user could adjust the view of the application along X and Y coordinates. Today Google Earth is a powerful application that is available over the internet. So what started as a concept and limited application years ago has evolved into An application for the masses. Check out the Google Earth video to see their latest capabilities. ArcView – ESRI is an application that combines geographic information with crime related information with the goal of improving law enforcement It supports various views so the user can locate locations related to crime types (e.g., burglaries, assaults, and threfts) The user can also view crime data by location and time (e.g., time of day, day of week, or month) The data can also be used to identify location relative to specific crimes (e.g., high car theft areas) The user can also Select type of information to display (roads, population, rainfall, topography, and political boundaries)
23
Direct Manipulation Interfaces
Video Games The most exciting, well-engineered, commercially successful application of direct-manipulation concepts? Pong Pacman ( Field of action is visual and compelling Button presses, joystick motions and knob rotations produce rapid response on the screen No syntax to remember Error messages are rare – the results of the action are obvious and easily reversed Often there is continuous display of the score (competition between others and the player themselves) Positive reinforcement that encourages mastery Perhaps The most exciting, well-engineered, and commercially successful applications have been video games. In the early days of computer gaming, the applications were very simple. Check-out the hyperlinks of Pong and Pacman to these groundbreaking games. Today 40 million Nintendo game players reside across 70% of those American households that include 8 to 12 year olds Some of the reasons for the appeal of video games include: First that the Field of action is visual and compelling Second, Button presses, joystick motions and knob rotations produce rapid response on the screen The user does not need to memorize any syntax With games Error messages are rare and the results of actions are obvious and easily reversed Often there is continuous display of the score (which promotes competition between others and the player themselves) Most games also provide Positive reinforcement that encourages mastery of the game
24
Direct Manipulation Interfaces
Video Games Educational Video Games – direct manipulation SimCity – education on urban planning The Sims – stronger attraction to women then men Games have also been designed to promote education. For example, SimCity has been used to educate students in urban planning. The Sims is a strategic life-simulation computer game. It is a simulation of the daily activities of one or more virtual persons ("Sims") in a suburban household near SimCity. It was first released in It has sold more than 6.3 million copies worldwide, making it the best-selling PC game in history.
25
Direct Manipulation Interfaces
Video Games Computer Role Playing Games (CRPGs) Players assume the role of a fictional character Activity takes place in a fictional world User’s control many of their character's actions Myst Massively Multiplayer On-Line Role Playing Games (MMORPGs) A large number of players interact with one another in a virtual world Interaction is in a persistent world hosted by the game's publisher continues to evolve while the player is away from the game Worldwide MMORPGs revenues exceeded half a billion dollars in 2005 World of Warcraft Now let’s focus on video games and specifically role playing games. Such games can be divided into two categories: Computer Role Playing Games (or CRPGs) and Massively Multiplayer On-Line Role Playing Games (or MMORPGs) In Computer Role Playing Games Players assume the role of a fictional character a in fictional world In the game the User takes control over many of that character's actions An example of such a game is Myst Myst is a graphic adventure video game. Myst puts the player in the role of the Stranger, who uses an enchanted book to travel to the island of Myst. There, the player uses other special books to travel to several worlds known as "Ages". Clues found in each of these Ages help reveal the back-story of the game's characters. The game has several endings, depending on the course of action the player takes. Myst was the best-selling PC game of all time, until The Sims exceeded its sales in 2002. In Massively Multiplayer On-Line Role Playing Games A large number of players interact with one another in a virtual world Interaction between players is in a persistent world hosted by the game's publisher The persistent world continues to exist and evolve while the player is away from the game In 2005 Worldwide revenues for MMORPGs exceeded half a billion dollars An example of this type of game is World of Warcraft World of Warcraft has more than 10 million monthly subscribers and is currently the world’s largest MMORPG.
26
Direct Manipulation Interfaces
Video Games versus Business Applications Game players Engaged in competition with a system or other players Seek entertainment and focus on challenge May prefer random events Application users Prefer a strong internal locus of control Focus on their tasks and may resent too many playful distractions Do not prefer random events If we contrast Game players and applications layers we find that Game players Are Engaged in competition with a system or other players They Seek entertainment and focus on challenge And gamers are more likely to prefer random events Application users Prefer a strong internal locus of control They Focus on their tasks and may resent too many playful distractions And they Do not prefer random events
27
Direct Manipulation Interfaces
Computer-aided Design Automobiles, electronic circuitry, aircraft, mechanical engineering Structural engineering, floorplans, interiors, landscaping, plumbing, electrical installation, etc. When the design is complete, the program can provide information regarding: Current Voltage drops Fabrication costs Manufacturing problems Computer-aided Design applications are used to design Automobiles, electronic circuitry, and aircraft, just to name a few applications. CAD applications are basically drafting tools. An example would be the electrical engineering design of a large building. The software Is interactive and can assist the user in assessing Current Voltage drops Fabrication costs and Manufacturing problems
28
Direct Manipulation Interfaces
Computer – aided Manufacturing and Process Control Honeywell’s Experion Process Knowledge System Provides the manager of a oil refinery or power utility plant with a colorized schematic of the plant Can indicate with a red line a sensor value that is out of range With a single click the operator can get a more detailed view of the troubling component A second click can provide more detailed information the sensor, or reset a value or circuit Basic strategy: eliminate the need for complex commands that the operator might only need to recall during a once-a-year emergency Computer-aided manufacturing (CAM) applications support engineers and machinists in manufacturing product components. CAM creates real life versions of components designed within a software package. Process Control pertains to user control of complex systems. We will discuss the details of process control later. But for now let’s consider an example of a direct manipulation user interface that the user depends on for the control of complex systems. Honeywell’s Experion Process Knowledge System Provides the manager of a oil refinery or power utility plant with a colorized schematic of the plant The system Can indicate with a red line a sensor value that is out of range With a single click the operator can get a more detailed view of the troubling component A second click can provide more detailed information the sensor, or reset a value or circuit Basic strategy: eliminate the need for complex commands that the operator might only need to recall during a once-a-year emergency
29
Direct Manipulation Interfaces
Direct Manipulation in Office Automation Xerox Star Sophisticated Text Formatting Graphics Multiple Fonts High Resolution Cursor Based Interface Apple Lisa Precursor to the Macintosh Hardware and software designs supported Pull-down menus Multi-window manipulation Editing of graphics and text Dragging of icons The birth of the first Direct Manipulation office computer was the Xerox Star in It was was capable of supporting Sophisticated Text Formatting Graphics Multiple Fonts High Resolution Cursor Based Interface The Apple Lisa followed in 1983 and was the Precursor to the apple Macintosh Hardware and software designs supported Pull-down menus Multi-window manipulation Editing of graphics and text Dragging of icons
30
Direct Manipulation Interfaces
Direct Manipulation in Office Automation MS-DOS Commands vs. Macintosh Direct Manipulation Tasks: Creating, copying, renaming, erasing files After training and practice, average task times: MS-DOS is 5.8 minutes Macintosh is 4.8 minutes After training and practice, average errors: MS-DOS is 2.0 Macintosh is 0.8 Since apple came out with a direct manipulation computer prior to Microsoft, there were empirical studies conducted to determine whether there were advantages to direct manipulation Over command entry. Tasks: Creating, copying, renaming, and erasing files After training and practice, average task times were: For MS-DOS is 5.8 minutes For Macintosh is 4.8 minutes After training and practice, average errors: For MS-DOS is 2.0 For Macintosh is 0.8 The results indicated a strong advantage to direct manipulation. Microsoft eventually brought the windows operating system to the market.
31
Direct Manipulation Interfaces
Continuing evolution of direct manipulation Quicken Home Automation Direct manipulation on a floor plan of: Burglar alarms Heat sensors Smoke detectors Opening/closing curtains or screens Air conditioning and heating Audio/video speakers or screens E.g., users can route sound from a MP3 player located in the living room to the kitchen by dragging the MP3 icon into the kitchen Virtual Worlds Travel through the human body Ride an electron cloud as it spins around a nucleus Let’s continue with a discussion regarding the evolution of direct manipulation Quicken – is a direct manipulation application for checkbook maintenance Homes are becoming more automated where the user has Direct manipulation of: Burglar alarms Heat sensors Smoke detectors Opening/closing curtains or screens Air conditioning and heating Audio/video speakers or screens E.g., users can route sound from a MP3 player located in the living room to the kitchen by dragging the MP3 icon onto a kitchen icon Direct manipulation also extends to Virtual World applications where the user can user point and click controls to navigate through through the human body Or Ride an electron cloud as it spins around a nucleus
32
Direct Manipulation Interfaces
Continuing evolution of direct manipulation Problems with direct manipulation Spatial or visual representation are not necessarily an improvement over text Especially for blind or visually impaired users Direct manipulation designs may consume considerable screen space May result in scrolling or multiple actions Users must learn the meanings of visual representations Icon interpretation For experienced typists, taking a hand off the keyboard to move a mouse may take more time then typing the relevant command Users may not share the same understanding of the metaphor, analogy, or conceptual model with the designer (testing is required) Browser based applications limit direct manipulation (e.g., drag & drop) Require Dynamic HTML, Java or Flash Problems with direct manipulation include: Spatial or visual representation are not necessarily an improvement over text Especially for blind or visually impaired users Direct manipulation designs may consume considerable screen space which May result in scrolling or multiple actions Users must learn the meanings of visual representations (e.g., having to interpret icons) For experienced typists, taking a hand off the keyboard to move a mouse may take more time then typing the relevant command Users may not share the same understanding of the metaphor, analogy, or conceptual model with the designer (testing is required to verify that users understand the metaphor) Browser based applications limit direct manipulation (e.g., drag & drop) which may Require complex development with Dynamic HTML, Java or Flash
33
Direct Manipulation Interfaces
Advantages of direct manipulation Continuous representation of the objects and actions of interest with meaningful visual metaphors Physical actions or presses of labeled buttons, instead of complex syntax Rapid, incremental, reversible actions whose effects on the objects of interest are visible immediately Advantages of direct manipulation include Having objects and actions Continuously represented with meaningful visual metaphors Allowing the user to make Physical actions or presses of labeled buttons, instead of having to enter complex syntax The support of Rapid, incremental, and reversible actions on objects that are immediately available
34
Direct Manipulation Interfaces
Advantages of direct manipulation Design systems with the following benefits Novices can learn basic functionality quickly Experts can work rapidly to carry out a wide range of tasks, even defining new functions and features Knowledgeable intermittent users can retain operational concepts Error messages are rarely needed Users can immediately see whether their actions are furthering their goals, and if the actions are counterproductive, they can change the direction of their activity Users experience less anxiety because the interface is comprehensible and because actions are easily reversed Users gain confidence and mastery because they are the initiators of action, they feel in control, and they predict the interface’s responses Additional Advantages of direct manipulation include: That Novices can learn basic functionality quickly And Experts can work rapidly to carry out a wide range of tasks Knowledgeable intermittent users can retain operational concepts Error messages rarely interrupt the task flow Users can immediately see whether their actions are furthering their goals, and if the actions are counterproductive, they can change the direction of their activity Users experience less anxiety because the interface is comprehensible and because actions are easily reversed Users gain confidence and mastery because they are the initiators of action, they feel in control, and they predict the interface’s responses
35
Direct Manipulation Interfaces
The OAI Model and Direct Manipulation The object of interest is displayed so that interface actions are close to the high-level task domain Little need for mental decomposition of tasks into multiple interface commands with complex syntactic forms (e.g., vi: go to line, go to word, go to character) Each action produces a comprehensible result in the task domain that is visible in the interface immediately The closeness of the task domain to the interface domain reduces operator problem-solving load and stress Compared to textual descriptors, visual representations of objects may be more natural and closer to human innate capabilities Action and visual skills emerged well before language in human evolution In The Object-Action Model (or OAI Model) The object of interest is displayed so that interface actions are close to the high-level task domain There is Little need for mental decomposition of tasks into multiple interface commands with complex syntactic forms (e.g., vi: go to line, go to word, go to character) Each action produces a comprehensible result in the task domain that is visible in the interface immediately (e.g., depress button when pressed) The closeness of the task domain to the interface domain helps to reduce the operator problem-solving load and decrease stress Compared to textual descriptors, visual representations of objects may be more natural and closer to human innate capabilities This is based on the premise that Action and visual skills emerged well before language in human evolution
36
Direct Manipulation Interfaces
Visual Thinking and Icons Semiotics – the study of signs and symbols Icon – an image, picture or symbol representing a concept In computer systems usually less than one inch square (64x64 pixels) Smaller icons are often integrated with a window border or toolbar Task dependency When working on a visual task (e.g., painting program), icons may be useful When working on a text-based task, it may be better to stay text based Icons with words (or mouse-overs) are useful Related to direct manipulation are the Visual cues communicated by icons The domain which studies signs and symbols is referred to as Semiotics An Icon is an image, picture or symbol representing a concept When working on a visual task icons may be useful (e.g., painting program) However, When working on a text-based task, it may be better to stay text based Mouse-overs can be used to display tool-tips (or words) when the mouse moves over the Icons
37
Direct Manipulation Interfaces
Icon related guidelines Represent the object or action in a familiar and recognizable manner Limit the number of different icons Make the icon standout from its background 3-d icons can be visually distracting Ensure that a single selected icon is clearly visible when surrounded by unselected icons Make each icon distinctive from every other icon Ensure harmoniousness of each icon as a member of a family of icons Design the movement animation (e.g., grayed-out ghost image on a drag) Detail information Larger shadowing for a larger file Color to show the age of a document Animation to show how much of a file has printed (document icon absorbed progressively into the printer icon) Explore use of combination of icons (E.g., drag a document to a printer icon) As we have discussed, direct manipulation interfaces lend themselves to the use of icons. Let’s discuss some Icon related guidelines First, Represent the object or action in a familiar and recognizable manner (e.g., a trash can icon to add files to a recycle bin. Interestingly, in the early days of the Macintosh computer, the user would drag the hard drive icon to the trash can to turn off the computer. Many users were concerned to do this because they felt that this action would delete the contents of their hard drive). Limit the number of different icons Icons should be noticeable, that is, Make the icons standout from their backgrounds 3-d icons can be visually distracting, so care should be taken in using 3-d effects Ensure that a single selected icon is clearly visible when surrounded by unselected icons (this can be seen in the diagram to the right) Make each icon distinctive from every other icon Ensure harmoniousness of each icon as a member of a family of icons (note the use of icons for font style as displayed in the diagram) Design an icon to indicate movement when using animation (e.g., grayed-out ghost image on a drag) It is possible to also using coding to communicate Detailed information For example, Larger shadowing for a larger file Coloring to show the age of a document Animation to show how much of a file has printed (e.g., a document icon being absorbed progressively into the printer icon) The UI designer could also Explore the use of combination of icons (or icon interactions, as to what event occurs when two icons touch) (E.g., drag a document to a printer icon)
38
Direct Manipulation Interfaces
Direct Manipulation Programming Example: programming a radio to a set of stations by pressing/holding a channel selection button Phone Services Direct manipulation can also be used to program devices or applications. For example, the user can program a radio to a set of stations by pressing and holding a channel selection button. The station is then mapped to that button. Visual programming languages refer tools where objects can be manipulated by the UI designer to create applications. The diagram displays an example of a direct manipulation programming language, where the user places icons into a flow diagram. Each icon performs a specific function. This was a application that I designed years ago where users could create phone services. In the example, the services begins by defining the Day of the week. That is, if the call is made to a specific number on the weekend, then the logic traverses one path, and if the call is made on a weekday, then the call traverses a different path. The service then divides the logic by time of day (work hours vs. Off-hours) , or from where the caller is calling from (Eastern US versus Western US). In some cases the call is routed to a phone, in other cases the call is routed to an announcement. The icons that are used to create each service is in the upper right of the diagram. The variables associated with each icon is in the lower left part of the diagram. Hence, with this application the user is able to program many different services by linking together icons, and not having to use a traditional programming language.
39
Direct Manipulation Interfaces
Direct Manipulation Programming Programming in the User Interface Sufficient computational generality Conditionals (if, then, else) Iteration (repeat/while) Access to appropriate data structures and operators File structures for directories Addition, subtraction, etc. Ease in programming By specification or by demonstration (Flash animation) Argument passing Simplicity in invocation and assignment of arguments Low risk High probability of bug free programs Halt and resume When creating direct manipulation Programming languages, the following principles apply: First, Sufficient computational generality must be in place (e.g., Conditionals (if, then, else) and Iterations (repeat/while)) Access to appropriate data structures and operators must be available (e.g., File structures for directories, Addition, subtraction, and other type operators). The programming should be made easy. For example, in some languages like Flash, the user moves objects on a stage, and the computer records their locations. As such animation becomes easy to program. Argument passing refers to passing information between objects (e.g., if a button is pressed then a new screen is displayed) Simplicity in invocation and assignment of arguments (e.g., simple events like pressing a mouse button, or typing a value in a field) Low risk (the tools should provide a High probability of bug free programs, and allow the designer to easily Halt and resume the program)
40
Direct Manipulation Interfaces
Direct Manipulation Programming Viscosity – the difficulty of making changes to a program Progressive evaluation – the capacity for execution of partial programs Viscosity refers to the difficulty of making changes to a program. Obviously for ease of use the viscosity should be low. Progressive evaluation refers to the the capacity for execution of partial programs (e.g., a designer may create a PC based program that when a link is pressed the browser is opened and a web page is displayed).
41
Direct Manipulation Interfaces
3-Dimensional Interfaces Some applications are designed as 2-D to be simpler than real-world systems Constrain movement Limit interface actions Ensure visibility of interface objects Enhanced 3D may be better than 3D Flying through objects Multiple simultaneous views of objects X-ray vision Shrink/expand objects Group/ungroup components Going back in time Less than successful 3D interfaces Air-traffic control Showing altitude by perspective drawing only adds clutter when compared to an overview from directly above 3-Dimensional Interfaces are another form of direct manipulation interfaces Some applications are better designed as 2-D. That is, 2-D is a simpler representation than real-world systems. Aspects of 2-D include: Movement is constrained to 2 dimensions as opposed to 3 dimensions 2-D Limits the number interface actions 2-D Ensures the visibility of the interface objects (i.e., objects cannot occlude other objects) Enhanced 3D may be better than 3D (which to some degree defy the laws of physics). Examples include: The ability to Fly through objects The ability to display Multiple simultaneous views of the same object X-ray vision Shrinking and expanding objects Grouping and ungroup components And Going back in time Attempts have been made to use 3-D interface for some applications, but data indicated that 2-D was the better approach: An Example is the Air-traffic control system (e.g., Showing altitude by perspective drawing only adds clutter when compared to an overview from directly above)
42
Direct Manipulation Interfaces
4-Dimensional Interfaces 4-D Anyone? Tesseract 3-Dimensional Interfaces are another form of direct manipulation interfaces Some applications are better designed as 2-D. That is, 2-D is a simpler representation than real-world systems. Aspects of 2-D include: Movement is constrained to 2 dimensions as opposed to 3 dimensions 2-D Limits the number interface actions 2-D Ensures the visibility of the interface objects (i.e., objects cannot occlude other objects) Enhanced 3D may be better than 3D (which to some degree defy the laws of physics). Examples include: The ability to Fly through objects The ability to display Multiple simultaneous views of the same object X-ray vision Shrinking and expanding objects Grouping and ungroup components And Going back in time Attempts have been made to use 3-D interface for some applications, but data indicated that 2-D was the better approach: An Example is the Air-traffic control system (e.g., Showing altitude by perspective drawing only adds clutter when compared to an overview from directly above)
43
Direct Manipulation Interfaces
Second Life Multi-user environment where users interact Users can choose avatars (fantasy images, desirable characteristics) Let’s look at some direct manipulation 3-D interfaces that are experiencing some success. Second Life is a Multi-user environment where users interact. Users are referred to as residents. Each resident is represented via an avatar. Residents can explore, meet residents, participate in individual and group activities, and create and trade items and services. Users move about the 3-D space by walking, running, and jumping. However, users can also fly, ride in vehicles, and teleport to locations. Residents can buy, sell, and rent land areas from each other. Second Life uses an internal currency called the "Linden Dollar" which are purchased with real dollars. Specific applications within Second Life include Education, Religion and the Arts (e.g., users can fly to the top of the Sistine Chapel to view the it up close in a manner that could not be done in the real world). At the beginning of September, 2008, just over 15 million accounts were registered, In January 2008, residents spent 28 million hours in second life. On average there are about 38,000 residents logged on at any particular moment.
44
Direct Manipulation Interfaces
3D Desktops and Workplaces Microsoft’s Task Gallery Intel’s Grand Canyon Xerox PARC’s Information Visualizer No successful products yet 3D Desktops and Workplaces have been design in an attempt to improve navigation and location of the PC desktop. In the example, the user can place objects in the 3D room, and click on those objects to have them activated. The example is Microsoft’s Task Gallery Intel’s Grand Canyon Xerox PARC’s Information Visualizer However, No successful products using 3D on the desktop have entered into the market yet.
45
Direct Manipulation Interfaces
Tips for effective 3D interfaces Use occlusion, shadows, perspective and other 3D techniques carefully Minimize the number of navigation steps for users to accomplish their tasks Keep text readable (better rendering, good contrast with background, an no more than 30-degree tilt) Avoid unnecessary visual clutter, distraction, contrast shifts and reflections Simplify user movement (keep movements planar, avoid surprises like going through walls) Organize groups of items in aligned structures to allow rapid visual search Enable users to construct visual groups to support spatial recall (e.g., placing items in corners) When designing 3D applications, the UI designer should consider the following: Use occlusion, shadows, perspective and other 3D techniques carefully Minimize the number of navigation steps for users to accomplish their tasks Keep text readable Avoid unnecessary visual clutter, distraction, contrast shifts and reflections Simplify user movement Organize groups of items in aligned structures to allow rapid visual search Enable users to construct visual groups to support spatial recall
46
Direct Manipulation Interfaces
Guidelines for inclusion of 3D features Provide users overviews so they can see the big picture Allow teleportation (rapid context shifts by selecting destination in an overview) Offer x-ray vision Provide history keeping (recording, undoing, replaying, editing) Permit rich user actions on objects (save, copy, annotate, share, send) Give users control over explanatory text (pop-up, floating, screen tips) Offer tools to select, mark and measure Implement dynamic queries to rapidly filter out unneeded items Enable landmarks to show themselves even at a distance Allow multiple coordinated views (users can be in more than one place at a time) In addition, the UI designer should: Provide users overviews so they can see the big picture Allow teleportation (rapid context shifts by selecting destination in an overview) Offer x-ray vision Provide history keeping (recording, undoing, replaying, editing) Permit rich user actions on objects (save, copy, annotate, share, send) Give users control over explanatory text (pop-up, floating, screen tips) Offer tools to select, mark and measure objects in the 3D space Implement dynamic queries to rapidly filter out unneeded items Enable landmarks to show themselves even at a distance Allow multiple coordinated views (users can be in more than one place at a time), so integrate those views
47
Direct Manipulation Interfaces
Teleoperation Derived from direct manipulation and process control Physical processes taking place in a remote location Clean-up in a nuclear reactor Need adequate feedback in sufficient time to permit effective decision making Manufacturing Medicine (consultation, radiology) Military operations (drones) Home automation Answering machines Security systems Energy control Appliances Welcome to Lesson 8 Part D Teleoperation is a type of application that has been Derived from direct manipulation and process control In Teleoperation applications the operator interacts with an application but the action takes place a remote location An example is the operation of robot that is used to Clean-up nuclear reactor waste For Teleoperation application adequate feedback must be provided in sufficient time to permit effective decision making. Other Teleoperation applications include Manufacturing Medicine (remote surgery) Military operations (e.g., operating drones to be shot down by air defense systems) Teleoperation is also used with Home automation Answering machines (e.g., checking messages remotely) Security systems (activating alarms remotely) Energy control (controlling temperature remotely) Appliances (starting dinner at home whlie note yet being at home)
48
Direct Manipulation Interfaces
Teleoperation Mahru Humanoid Robot Real-Time Teleoperation Dominos Drones on 60 Minutes Welcome to Lesson 8 Part D Teleoperation is a type of application that has been Derived from direct manipulation and process control In Teleoperation applications the operator interacts with an application but the action takes place a remote location An example is the operation of robot that is used to Clean-up nuclear reactor waste For Teleoperation application adequate feedback must be provided in sufficient time to permit effective decision making. Other Teleoperation applications include Manufacturing Medicine (remote surgery) Military operations (e.g., operating drones to be shot down by air defense systems) Teleoperation is also used with Home automation Answering machines (e.g., checking messages remotely) Security systems (activating alarms remotely) Energy control (controlling temperature remotely) Appliances (starting dinner at home whlie note yet being at home)
49
Direct Manipulation Interfaces
Design to accommodate teleoperation issues Slow response times and time delays Transmission delay (time for command to reach the microscope) Operation delay (time until the microscope responds) Incomplete feedback The microscope can transmit its current position, but operates so slowly it cannot indicate the exact current position Unanticipated interferences The slide is accidentally moved by a person at the local site May be better for the user to specify a destination (rather than a motion) and wait until the action is completed Potential problems related to teleoperation include: Slow response times and time delays Transmission delay refers to the time from when the command is issued to the time the feedback is obtained (e.g., from the time the user adjust a control on a microscope to the time the image appears in the lens) Operation delay in this example would refer to the time until the microscope responds The second problem related to teleoperation is Incomplete feedback (e.g., The microscope can transmit its current position, but operates so slowly it cannot indicate the exact current position) The third problem pertains to Unanticipated interferences (e.g., The slide is accidentally moved by a person at the local site) The user interface design May be better for the user to specify a destination (rather than a motion) and wait until the action is completed
50
Direct Manipulation Interfaces
Telemedicine Remote examination Remote surgery Telepathology Magnification Focus Illumination Position Telemedicine is practiced using two technologies: real time (synchronous) and store-and-forward (asynchronous). An example of Real time telemedicine is robotic surgery. Video-conferencing is another form of synchronous telemedicine. There are peripheral devices which can be attached to the video-conferencing equipment. For instance, a tele-otoscope allows a remote physician to 'see' inside a patient's ear; a tele-stethoscope allows the remote physician to hear the patient's heartbeat. Many Medical specialties use telemedicine. While I worked for BellSouth I was involved in a telemedicine project where psychiatric care was provided from the UNC medical school to patients in nursing homes in the outer banks. One patient remarked that telemedicine was the best invention since “screen-wire”. Screen-wire meant screens that kept insects out of homes and hospitals to reduce infection. The model used by UNC was patient centric. That is, instead of moving the patient to the location where they would receive care, let the patient stay in their location and bring medical care to the patient via telemedicine. Store-and-forward telemedicine involves acquiring medical data (like medical images) and then transmitting this data to a doctor assessment offline. Telepathology needs to provide the following capabilities to the remote physician: Magnification Focus Illumination Position
51
Direct Manipulation Interfaces
Virtual and Augmented Reality “Being in” as opposed to “Looking at” Architectural applications Wall-sized image to give perspective Animation to simulate movement (left to right) Treadmill to simulate walking toward, walk through doors, stairs Replace projector with a head-mounted display Some applications are better when “looked at” Air-traffic control To surgeons want to “Be in” the patients body? Training using virtual reality Virtual and Augmented Reality involves “Being in the application” as opposed to “Looking at the application” Architectural applications include Wall-sized image to give perspective Animation to simulate movement Treadmills to simulate walking toward, walk through doors, stairs Replace projector with a head-mounted display Some applications are better when “looked at” (e.g., Air-traffic control) Training is a very common use of virtual reality You may want to check out some of the example by Fifth Dimension Technologies
52
Direct Manipulation Interfaces
Virtual and Augmented Reality The CAVE (National Center for Supercomputing Applications) An immersive virtual reality facility designed for the exploration of and interaction with spatially engaging environments. The stereoscopic capabilities, coupled with its uniquely immersive design, enable scientists and researchers to interact with their data An atmospheric scientist can actually "climb inside" of a hurricane and visualize its complex and chaotic elements from any angle or visual perspective A biological researcher, examining a tightly coiled strand of DNA, can virtually "unravel" this strand and manipulate it in an environment that preserves the critical depth information of the data. Teach a child to cross a street An example of Virtual and Augmented Reality Is The CAVE (which was developed by National Center for Supercomputing Applications) CAVE is an immersived virtual reality facility designed for the exploration of and interaction with spatially engaging environments. The stereoscopic capabilities, coupled with its uniquely immersive design, enable scientists and researchers to interact with their data An atmospheric scientist can actually "climb inside" of a hurricane and visualize its complex and chaotic elements from any angle or visual perspective A biological researcher, examining a tightly coiled strand of DNA, can virtually "unravel" this strand and manipulate it in an environment that preserves the critical depth information of the data.
53
Direct Manipulation Interfaces
Artificial Reality VideoPlace – Myron Krueger Surround the user with an artificial reality which responded to their movements and actions. The users were able to visually see the results of their actions on screen, through the use of colored silhouettes. The users had a sense of presence while interacting with onscreen objects and other users. The sense of presence was enough that users pulled away when their silhouettes intersected with those of other users. eHarmony 2.0? An example of an Artificial Reality application is VideoPlace by Myron Krueger The concept was to surround the user with an artificial reality which responded to their movements and actions. The users were able to visually see the results of their actions on screen, through the use of colored silhouettes. The users had a sense of presence while interacting with onscreen objects and other users. The sense of presence was enough that users pulled away when their silhouettes intersected with those of other users.
54
Direct Manipulation Interfaces
Applications of virtual environments Phobia treatment Acrophobia Pain Control Immersive environments provide distractions for patients Interior Design OAI model Click, drag, enlarge objects Room painting tool There are many practical problems solved by way of the use of virtual environment applications e.g., Phobia treatment such as Acrophobia (or fear of heights) And Pain Control where Immersive environments provide distractions for patients Interior Design is another area were virtual environment applications are used Based on the OAI model users can Click, drag, enlarge objects (e.g., furniture, window treatments or room painting) (just the walls, all other objects not modified)
55
Direct Manipulation Interfaces
Augmented Reality See the real world with an overlay of additional information See wires or plumbing behind walls Tourist glasses – label buildings in a historic town Molecular biology Augmented Reality Allows the user to See the real world with an overlay of additional information e.g., the user could See wires or plumbing behind walls Or Tourist glasses could label buildings in a historic towns In the diagram, a Molecular biologist can physically manipulate a model of a molecule, and have it interact with other molecules displayed on the computer screen.
56
Direct Manipulation Interfaces
Virtual environments dependent on integration of multiple technologies Visual Display Normal Display 12 to 17 inches diagonally at a normal viewing distance of 70 cm subtends a visual angle of about 5-degrees Large Screen 17 to 30 inches can cover 20 to 30-degrees Head Mounted Displays 100 degrees horizontally and 60-degrees vertically Head motion produces new images so users perceive 360-degrees Displays must approach 100-millisecond delay in presenting images to approach real time Virtual environments are dependent on the integration of multiple technologies First is the Visual Display On a Normal Display a 12 to 17 inch diagonal screen at a normal viewing distance of 70 cm (about 27 inches) subtends a visual angle of about 5-degrees On a Large Screen 17 to 30 inch diagonal screen can cover subtends a visual angle of about 20 to 30-degrees providing a more immersed user experience Most virtual environment use Head Mounted Displays Where the image allows a field of view of 100 degrees horizontally and 60-degrees vertically Each Head motion produces a new image so users perceive 360-degrees Displays must approach 100-millisecond delay in presenting images to approach real time
57
Direct Manipulation Interfaces
Virtual environments dependent on integration of multiple technologies Head-position sensing Head-mounted displays can provide differing views depending on head position Hand-position sensing DataGlove The first diagram displays an example of a head-mounted display. The second diagram is a data glove where sensors detect where the user is pointing and what the user is touching in virtual world
58
Direct Manipulation Interfaces
Virtual environments dependent on integration of multiple technologies Force feedback and haptics Hand-operated remote-control devices for performing chemistry experiments or for handling nuclear materials Gives users a sense of grasp Sound input and output Training of Army tank crews while using realistic sounds of battle resulted in: Elevated heart rates, more rapid breathing, and increased perspiration Speech recognition for initiating actions and making menu selection Keyboard and mouse use is restricted Other sensations Tilting and vibration of flight simulators Collaborative and competitive virtual environments Two people at remote sites working together while seeing each others actions and the object of interest Additional technologies that must be supported to sustain a true immersed experience include: Force feedback and haptics where users experience a sense of grasp (e.g., Hand-operated remote-control devices for performing chemistry experiments or for handling nuclear materials) Sound input and output is another technology (e.g., Training of Army tank crews while using realistic sounds of battle resulted in: Elevated heart rates, more rapid breathing, and increased perspiration: evidenced of a more immersed experience) Speech recognition for initiating actions and making menu selection where the user is restricted from using the Keyboard or mouse Other sensations include Tilting and vibration of flight simulators Finally, the concept of Collaborative and competitive virtual environments involves Two people at remote sites working together while seeing each others actions and the object of interest
59
Direct Manipulation Interfaces
Piaget’s Four Stages of Development Sensorimotor (birth to 2 years) Preoperational (2 to 7 years) Concrete operational (7 to 11 years) Physical actions on an object are comprehensible Children acquire the concept of conservation or invariance Direct manipulation brings activity to this stage Formal operational (begins at 11 years) Symbol manipulation to represent actions on objects One interesting final note, is that when an application is targeted for a specific segment of the population, the UI designer needs to consider the special needs of the users. In this case we look at children as a segment. According to Piaget’s Four Stages of Development Stage 1 is Sensorimotor (birth to 2 years) Stage 2 is Preoperational (2 to 7 years) Stage 3 Concrete operational (7 to 11 years) Physical actions on an object are comprehensible Children acquire the concept of conservation or invariance Direct manipulation brings activity to this stage Hence at this stage direct manipulation becomes more powerful an interface model Stage 4 Formal operational (begins at 11 years) Symbol manipulation to represent actions on objects For designing applications for children, the UI designer should look more closely at the mental and physical capabilities as a function of age.
60
Future Direct Manipulation
Tangible User Interface (TUI) Based on the physical embodiment of digital information & computation Went beyond the dominant paradigm of “Painted Bits” (GUI). Humans have evolved a heightened ability to sense and manipulate the physical world TUIs expand the affordances of physical objects, surfaces, and spaces so they can support direct engagement with the digital world. Radical Atoms Assume a hypothetical generation of materials that can change form and appearance dynamically, becoming as reconfigurable as pixels on a screen. Radical Atoms are coupled with bits so that dynamic changes of physical form can be reflected in digital states in real time, and vice versa. It is likely that Future Interfaces will include the following New devices which are portable, inexpensive, small, wearable, mobile, and personal Context-aware devices to provide location based information Perceive user needs perhaps related to health care for example Small medical sensors that monitor health Hidden detectors that protect from dangers Visual, aural, tactile, gestural interaction with devices are likely to increase in the next few years Universal usability to improve voting, and crime reporting Biometric identification to reduce the chance of terrorism
61
Future Direct Manipulation
Tangible Bits (TUI) - a user interface where a person interacts with digital information through the physical environment. It is likely that Future Interfaces will include the following New devices which are portable, inexpensive, small, wearable, mobile, and personal Context-aware devices to provide location based information Perceive user needs perhaps related to health care for example Small medical sensors that monitor health Hidden detectors that protect from dangers Visual, aural, tactile, gestural interaction with devices are likely to increase in the next few years Universal usability to improve voting, and crime reporting Biometric identification to reduce the chance of terrorism
62
Future Direct Manipulation
Tangible Media - inFORM - Interacting With a Dynamic Shape Display It is likely that Future Interfaces will include the following New devices which are portable, inexpensive, small, wearable, mobile, and personal Context-aware devices to provide location based information Perceive user needs perhaps related to health care for example Small medical sensors that monitor health Hidden detectors that protect from dangers Visual, aural, tactile, gestural interaction with devices are likely to increase in the next few years Universal usability to improve voting, and crime reporting Biometric identification to reduce the chance of terrorism
63
Future Direct Manipulation
zSpace – 3D Immersive Technology Gesture Based Interaction – e.g., head position It is likely that Future Interfaces will include the following New devices which are portable, inexpensive, small, wearable, mobile, and personal Context-aware devices to provide location based information Perceive user needs perhaps related to health care for example Small medical sensors that monitor health Hidden detectors that protect from dangers Visual, aural, tactile, gestural interaction with devices are likely to increase in the next few years Universal usability to improve voting, and crime reporting Biometric identification to reduce the chance of terrorism
64
Future Direct Manipulation
Aireal – Tactile 3D Haptic technology that delivers tactile sensations in mid air. AIREAL enables users to feel virtual objects, experience dynamically varying textures and receive feedback on full body gestures It is likely that Future Interfaces will include the following New devices which are portable, inexpensive, small, wearable, mobile, and personal Context-aware devices to provide location based information Perceive user needs perhaps related to health care for example Small medical sensors that monitor health Hidden detectors that protect from dangers Visual, aural, tactile, gestural interaction with devices are likely to increase in the next few years Universal usability to improve voting, and crime reporting Biometric identification to reduce the chance of terrorism
65
Future Direct Manipulation
Forget devices; the future of technology is seeded in biology Will completely healthy people start modifying their bodies to make themselves harder, better, faster, stronger? Tattoo a smartphone microphone onto your throat Captures vibrations directly from your larynx in order to cut out background noise It is likely that Future Interfaces will include the following New devices which are portable, inexpensive, small, wearable, mobile, and personal Context-aware devices to provide location based information Perceive user needs perhaps related to health care for example Small medical sensors that monitor health Hidden detectors that protect from dangers Visual, aural, tactile, gestural interaction with devices are likely to increase in the next few years Universal usability to improve voting, and crime reporting Biometric identification to reduce the chance of terrorism
66
Direct Manipulation Computer mouse inventor Douglas Engelbart dies – July 2013 It is likely that Future Interfaces will include the following New devices which are portable, inexpensive, small, wearable, mobile, and personal Context-aware devices to provide location based information Perceive user needs perhaps related to health care for example Small medical sensors that monitor health Hidden detectors that protect from dangers Visual, aural, tactile, gestural interaction with devices are likely to increase in the next few years Universal usability to improve voting, and crime reporting Biometric identification to reduce the chance of terrorism
67
Direct Manipulation Interfaces
Meta glasses bring 3D and your hands into the picture It is likely that Future Interfaces will include the following New devices which are portable, inexpensive, small, wearable, mobile, and personal Context-aware devices to provide location based information Perceive user needs perhaps related to health care for example Small medical sensors that monitor health Hidden detectors that protect from dangers Visual, aural, tactile, gestural interaction with devices are likely to increase in the next few years Universal usability to improve voting, and crime reporting Biometric identification to reduce the chance of terrorism
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.