Input and Interaction Lecture No. 4

Introduction Interactive computer graphics opens up a myriad of applications, ranging from interactive design of buildings, to control of large systems through graphical interfaces, to virtual-reality systems, to computer games.

Interaction One of the most important advances in computer technology was enabling users to interact with computer displays. Ivan Sutherland’s Project Sketchpad launched the present era of interactive computer graphics.

Interaction OpenGL does not support interaction directly. The major reason for this omission is that the system architects who designed OpenGL wanted to increase its portability by allowing the system to work in a variety of environment Windowing and input functions were left out of the Application Programming Interface (API.s).

Interaction We can avoid such potential difficulties by using a simple library, or toolkit. The toolkit can provide the minimal functionality, such as opening of windows, use of the keyboard and mouse, and creation of pop-up menus through the toolkit’s API.

Input Devices Physical vs. logical Physical Logical mouse, keyboard, etc. perspective of how they interact with system Logical function perspective of what they send to the application familiar to all writers of high-level programs

Input Devices For example, data input and output in C are done through functions such as printf, scanf, getchar, and putchar, whose arguments use the standard C data types, and through input (cin) and output (cout) streams in C++.

Input Devices In computer graphics, the use of logical devices is more complex , because the forms that input can take are more varied than the strings of bits or characters to which we are usually restricted in non-graphical applications.

Input Devices For example, we can use the mouse—a physical device— either to select a location on the screen, or to indicate which item in a menu we wish to select. In the first case, an x, y pair (in some coordinate system) is returned to the user program; in the second, the application program may receive an integer as the identifier of an entry in the menu.

Physical Input Devices Two categories: pointing devices: allows the user to indicate a position on the screen. keyboard devices: a physical keyboard, or generally, include any device that returns character codes to a program.

Physical Input Devices Absolute-positioning ( some fixed position) pen returns position (settable) data glove always returns 3D position coords Relative-positioning ( current position) mouse pos. always begins where cursor is

Measure and Trigger The manner by which physical and logical input devices provide input to an application program can be described in terms of two entities: a measure process: is what the device returns to the user program. a device trigger: is a physical input on the device with which the user can signal the computer.

Input modes Three distinct mode for a measure of a device. Each mode is defined by the relationship between the measure process and the trigger Request mode: the measure of the device is not returned to the program until the device is triggered. This input mode is standard in non graphical applications, such as a typical C program that requires character input.

The relationship between measure and trigger for Request mode Trigger Request Measure The relationship between measure and trigger for request mode Measure Process Program Trigger Process

Inputs modes Sample-mode: input is immediate , as soon as the function call in the user program is encountered, the measure is returned. No trigger is needed Measure Measure Process Program

Inputs modes In both request- and sample-mode the user must identify which device is to provide the input. We usually interface with the devices through functions such as request_locator(device_id, &measure); sample_locator(device_id, &measure);

Inputs modes Both request and sample modes are useful for situations where the program guides the user, but are not useful in applications where the user controls the flow of the program. For example , a flight simulator might have multiple input devices—such as a joystick, buttons, and switches. Sample- and request-mode input are not sufficient for handling the variety of possible human–computer interactions that arise in a modern computing environment.

Inputs modes Event mode: the most flexible input mode triggers generate events (events store data) measures are sent to 1) an event queue program checks queue events are examined and acted upon 2) a special-purpose function program associates function called a callback used with the major windowing systems because it has been proved to work well in client–server environments.

Event - Mode Model Trigger Measure A wait Event Trigger Process Program

Event-driven input Callback for display Callback for mouse glutDisplayFunc(display) Callback for mouse glutMouseFunc(mouse) Callback for keyboard glutKeyboardFunc(keyboard);

Clients and servers In OpenGL In general: Servers perform tasks for clients In OpenGL assumes the OpenGL program is the client assumes workstations are graphics servers that provide display and input services to the OpenGL program for example, the client can display its output on any networked server

Display lists Display list is a group of OpenGL commands that have been stored (compiled) for later execution. Once a display list is created, all vertex and pixel data are evaluated and copied into the display list memory on the server machine. It is only one time process. After the display list has been prepared (compiled), you can reuse it repeatedly without re-evaluating and re-transmitting data over and over again to draw each frame.

Display lists Display list is one of the fastest methods to draw static data because vertex data and OpenGL commands are cached in the display list and minimize data transmissions from the client to the server side. It means that it reduces CPU cycles to perform the actual data transfer. Another important capability of display list is that display list can be shared with many clients, since it is server state.

Display lists We can send graphical entities to a display in one of two ways: immediate mode: send the complete description of our objects (vertices, attributes, and primitive types )to the graphics server. No memory ,re-compute to redisplay.

Display lists retained-mode: define the object once, then put its description in a display list. The display list is stored in the server and redisplayed by a simple function call issued from the client to the server.

Display lists Advantage: reduced network traffic. allows the client to take advantage of any special-purpose graphics hardware that might be available in the graphics server. Disadvantage: require memory on the server. overhead of creating a display list.

Display lists Create multiple lists glGenLists(number) generates multiple lists glCallLists displays multiple lists

Menus GLUT provides pop-up menus, that we can use with the mouse to create sophisticated interactive applications. Using menus involve taking a few simple steps: - define entries -link menu to mouse button - define callback function for each menu entry

Picking Picking is the logical input operation that allows the user to identify an object on the display. Although the action of picking uses the pointing device, the information that the user wants returned to the application program is not a position. Old display processors could accomplish picking easily by means of a lightpen.

Picking One reason for the difficulty of picking in modern systems is the forward nature of their rendering pipelines. Primitives are defined in an application program and move forward through a sequence of transformations and clippers until they are rasterized into the frame buffer.

Picking three ways to deal with this difficulty: selection: involves adjusting the clipping region and viewport such that we can keep track of which primitives in a small clipping region are rendered into a region near the cursor. These primitives go into a hit list that can be examined later by the user program.

Picking bounding rectangles: or extents, for objects of interest. The extent of an object is the smallest rectangle, aligned with the coordinates axes, that contains the object. back buffer: Recall that the back buffer is not displayed and thus we can use it for purposes other than forming the image we will display when we swap the front and back buffers.

Buffering Single buffering Double buffering objects are always rendered into the same frame buffer, which is always being displayed delay of clearing of and re-drawing into frame-buffer will cause flicker if re-draw takes longer than refresh or refresh and animation not synced The higher the refresh rate, the less "flicker" you will see. Double buffering keep two buffers (front and back) always display front, always render into back* swap front and back when rendering complete