1. Past, Present and Future of User Interface Software Tools
Brad A. Myers
(work with Scott E. Hudson and Randy Pausch)
Human-Computer Interaction Institute
School of Computer Science
Carnegie Mellon University

2. Introduction User Interface Software Tools
Help developers design and implement user interfaces
Focus on Tools, but influenced by future UIs
Today’s tools are highly successful
Window Managers, Toolkits, Interface Builders ubiquitous
Most software built using them
Are based on HCI research Brad A. Myers. “A Brief History of Human Computer Interaction Technology.” ACM interactions. Vol. 5, no. 2, March, pp
Future tools must be different

3. What Worked Window Managers and Toolkits Event Languages
Graphical, Interactive Tools
Component Architectures
Hypertext
Object Oriented Programming

4. Window Managers
Multiple (tiled) windows in research systems of 1960’s: NLS, etc.
Overlapping introduced in Alan Kay’s thesis (1969)
Smalltalk, 1974 at Xerox PARC
Successful because multiple windows help users manage scarce resources:
Screen space and input devices
Attention of users
Affordances for reminding and finding other work

5. Toolkits A collection of widgets (“controls”)
Menus, scroll bars, text entry fields, buttons, etc.
Toolkits help with programming
Help maintain consistency among UIs
Key insight of Macintosh toolbox
Successful partially because address common, low-level features for all UIs

6. Event Languages Create programs by writing event handlers
Many UIMSs used this style
Univ. of Alberta (1985), Sassafras (1986), etc.
Now used by HyperCard, Visual Basic, Lingo, etc.
Toolkits with call-backs or action methods are related
Advantages:
Natural for GUIs since generate discrete events
Flow of control in user’s hands rather than programmer’s
Discourages moded UIs
Won’t work well in future

7. Graphical Interactive Tools
Create parts of user interface by laying out widgets with a mouse
Examples: Menulay (1983), Trillium (1986), Jean-Marie Hullot from INRIA to NeXT
Now: HyperCard, Interface Builders, Visual Basic’s layout editor, resource editors, “constructors”
Advantages:
Graphical parts done in an appropriate, graphical way
Accessible to non-programmers

8. Component Architectures
Create applications out of components which are separately developed and compiled
In UI software, each component controls an area of the screen
Example: drawing component handles picture inside a document
Invented by Andrew research project at CMU (1988)
Now: OLE, OpenDoc, ActiveX, Java Beans

9. Hypertext
Ted Nelson named it in 1965 and developed a Hypertext system at Brown University
Important systems: NLS (1967), Hyperties (1986)
World-Wide Web
Phenomenal success due to:
Ease of use of Mosaic browser
Support for embedded graphics
Support for easy authoring
Tim Berners-Lee used the hypertext idea to create the World Wide Web in 1990 at the government-funded European Particle Physics Laboratory (CERN).

10. Object Oriented Programming
Success of OO owes much to UI software field
Popularized by Smalltalk
GUI elements (widgets) seem like objects
Have state, accept events (messages)
Rise parallels GUIs
C++ with Windows 3.1

11. What Hasn’t Caught On User Interface Management Systems
Separation of UI and application is harder than expected
Formal Language-Based Tools
Grammars and Transition networks are not appropriate for direct manipulation
Constraints
Declare once, system maintains
Get to be unpredictable and hard to control
Model-Based and Automatic Techniques
Results are not as good as hand-created
Sometimes, no savings of effort

12. Future Prospects and Visions
Important Trends
Ubiquitous Computing
Move to recognition-based interfaces
End-user customization and scripting

13. Ubiquitous Computing Computation embedded in many kinds of devices
Digital pagers and cell phones, Palm Pilots, CrossPads, laptops, wall-size displays, “smart” rooms
Next wave: easy communication with radio
E.g., BlueTooth:
Significant Implications for tools
Moving target problem
Tools for coordinating multiple, distributed, communicating devices
“Multi-computer” user interfaces

14. More problems: Varying Input and Output
Today’s Desktop screens vary by a factor of 2.5 in size and a factor of 4 in pixels
Tomorrow’s screen will vary by factors of 100 in size and a factor of 625 in pixels
Cell phone to Stanford’s wall (3796 x 1436 pixels)

15. Varying I/O Capabilities, cont.
Interaction techniques for desktop will not work
No room on small devices
Can’t reach menubar on wall-size devices
Want to run same application on different devices

16. Need for Prototyping Devices
User interface will be in hardware
Rapid design and prototyping needed for hardware
Pragmatics and usability cannot be evaluated from a simulation on a screen

17. Move to recognition-based interfaces
Speech, gestures, camera-based vision
Multimodal interaction
User will pick which modality to use
Use multiple modalities at same time
Today, programming these requires knowing about Hidden-Markov Models, grammars, feature vectors, etc.
Need tools to hide these complexities

18. Fundamental Differences of Recognition-based UIs
Input is uncertain
Recognition can make errors
Requires monitoring, feedback, correction
Do not produce discrete events
GUI event model no longer works
Interpreting input requires deep knowledge of data
Context of the application
“Move the red truck to here”
Need a architecture based on accessible application data structures
“Reflection”, “Open Data Model”

19. More Assumptions of Today’s Tools
Skill and Dexterity of users
Older users
Makes single, fixed library of widgets untenable
Non-overlapping and opaque components
Preclude translucency, magic lens interactions

20. Operating Systems Considerations
What is in the OS?
Window Manager? Toolkit? Communication? Scripting facilities?
Need ever increasing services for applications
Need more access to low-level information
E.g., hardware buttons, whether on network
Ideally, API to support competition and research into these components

21. Conclusions
Research in tools necessarily trails innovation in UI design
Due to consolidation on desktop metaphor, significant progress in tools
UI design poised for radical changes
New opportunities and challenges for tools

22. Past, Present and Future of User Interface Software Tools
Thank you!
Past, Present and Future of User Interface Software Tools
Brad A. Myers
(work with Scott E. Hudson and Randy Pausch)
Human-Computer Interaction Institute
School of Computer Science
Carnegie Mellon University

23. Talk given with SlideShow Commander
Result of the Pebbles research project
SlideShow Commander to be commercialized by:
Available soon!
Synergy Solutions, Inc.
Brad Myers 23