1. Cs 352:
Interactive
2D and 3D
Computer Graphics

2. This Class Interactive 2D and 3D Graphics Programming
Interactive Computer Graphics
This Class
Interactive 2D and 3D Graphics Programming
(with a taste of photorealistic graphics, image processing, and modeling)
Top-down approach
Course Information
Syllabus
Policies
Platform
Projects

3. Aspects of Graphics Design vs. Programming
Interactive Computer Graphics
Aspects of Graphics
Design vs. Programming
Interactive vs. Photorealistic
2D vs. 3D
Graphics vs. image processing vs. user interfaces

4. Kinds of Graphics Software
Interactive Computer Graphics
Kinds of Graphics Software
Photoshop, Illustrator, etc.
3D Modeling (CAD, animation)
Rendering (ray tracing, radiosity)
Animation tools
Graphics programming APIs (OpenGL, DirectX)
Scene graph libraries
Game engines
3D modeling programs: 3D Studio Max, Lightwave, Maya, Blender
Rendering: POV Ray, RenderMan, many others
Animation: Maya, Lightwave, Blender, many others

5. Comet Simulation Interactive Computer Graphics
COMET CRASH - Sandia supercomputer simulations of a one-kilometer comet entering Earth's atmosphere, approaching the ocean's surface, and impacting the ocean, deforming the ocean floor and creating a giant high-pressure steam explosion rising into the stratosphere. The explosion ejects comet vapor and water vapor into ballistic trajectories that spread around the globe. The New York City skyline is shown for scale.
Graphics can make for compelling communication.

6. Ray-traced Image Interactive Computer Graphics
Ray-traced images can be near photo-realistic.

7. Interactive Computer Graphics
New ways to communicate: changes the world.

8. OpenGL OpenGL: a widely-used, open API for 3D graphics Support
Interactive Computer Graphics
OpenGL
OpenGL: a widely-used, open API for 3D graphics
Old, originally from Silicon Graphics (SGI)
Low-level, procedural (vs. scene graph retained mode)
Designed for speed, control over hardware
Need hardware support for top performance
Widely used for CAD, VR, visualization, flight simulators
Managed by non-profit “Khronos Group” consortium
Support
All major graphics cards, platforms have support
Mobile devices (iOS, Android) use an embedded version
HTML5 has experimental WebGL support
Bindings for JavaScript, Java, C#, Perl, Python, Ruby, Scheme, Visual Basic, Ada, …

9. Graphics Only OpenGL does not support windowing, interaction, UI, etc
Interactive Computer Graphics
Graphics Only
OpenGL does not support windowing, interaction, UI, etc
It must be used with another windowing system/library such as
MS Windows—various
Cocoa
X11 Qt
GLUT, GLFW
HTML5 JavaScript?
GLUT, GLFW: simple, fast, cross-platform windowing for OpenGL

10. History GL (SGI), 1980s to early 1990s [reality engine?]
Interactive Computer Graphics
History
GL (SGI), 1980s to early 1990s [reality engine?]
OpenGL Architecture Review Board, 1992
Selected versions:
1.0, 1992 (Happy Twentieth birthday!)
1.3, 2001—better texture support
2.0, 2004—GLSL (GL Shading Language, user programmable vertex shaders)
3.0, 2008—plan: fundamental changes to the API—no longer state-based, requires use of GLSL 1.3. Compromise: old API deprecated (but still used)
4.1, 2010—new geometry control, shaders, OpenGL ES 2.0 compatibility

11. Interactive Computer Graphics
OpenGL ES
OpenGL ES (for Embedded Systems) is a subset of OpenGL for mobile phones, consoles, etc
Common and Common Lite profiles (lite profiles are fixed-point only)
Version 2.0 released in 2007
GLSL for shaders
Supported in iOS, Android, WebGL,…
Version 3.0, 2012: texture compression, new version of GLSL ES, 32-bit floats, enhanced texturing

12. Interactive Computer Graphics

13. Refraction using vertex shaders
Interactive Computer Graphics
Refraction using vertex shaders
OpenGL with programmable shaders: can get beyond what used to be normal for graphics shaders: e.g. refraction.

14. OpenGL vs. proprietary OpenGL DirectX: Metal (Apple) Older
Interactive Computer Graphics
OpenGL vs. proprietary
OpenGL
Older
Has survived the Direct3D challenge and emerged as undisputed standard for 3D graphics programming
Used more for professional applications
Mobile gaming is mostly on OpenGL ES
Unreal, Unity, other game engines on OpenGL ES
DirectX:
MS only
Used more for games
Latest versions are good
Metal (Apple)

15. WebGL OpenGL 2.0 ES in your Web browser, no plugins needed!
Interactive Computer Graphics
WebGL
OpenGL 2.0 ES in your Web browser, no plugins needed!
Supported by all major browsers except IE (Microsoft hates Web standards, OpenGL)
Working group: Apple, Google, Mozilla, Opera (not Microsoft)

16. Interactive Computer Graphics

17. Other software we’ll use
Interactive Computer Graphics
Other software we’ll use
POV ray-tracer
ImageMagick image manipulation library
3D Modeling: Google's SketchUp or Blender
HTML5 Canvas element for 2D graphics
The only cross-platform environment nowadays…
Overview
Three.js graphics library for WebGL

18. Chapter 1: Graphics Systems and Models
Interactive Computer Graphics
Chapter 1: Graphics Systems and Models
A Graphics System
Processor
Memory
Frame Buffer
Display
Input Devices
Output Devices
Graphics processor: move graphics computatation to s separate, specialized processor

19. Images Array of pixels Red, Green, Blue
Interactive Computer Graphics
Images
Array of pixels
Red, Green, Blue
May also have an alpha value (opacity)
Why red, green, and blue? Is that what color really is?

20. Pixels and the Frame Buffer
Interactive Computer Graphics
Pixels and the Frame Buffer
Pixels:
picture elements
3 values: RGB, or or
4 values: RGBA (Alpha = opacity)
Frame buffer
Depth: bits per pixel
May have 24, 32, 64, or flexible depth
HDR
High dynamic range: 12 or more bits per color

21. The Human Visual System
Interactive Computer Graphics
The Human Visual System
Rods: night vision
Cones: day vision
Three types of cones, with different color sensitivity
We model and render for its capabilities

22. Interactive Computer Graphics

23. Spectral Sensitivity Color spectrum: 780 nm (blue)…350 nm (red)
Interactive Computer Graphics
Spectral Sensitivity
Color spectrum: 780 nm (blue)…350 nm (red)

24. Interactive Computer Graphics

25. Display terms Scan line Resolution Horizontal and vertical re-trace
Interactive Computer Graphics
Display terms
Scan line
Resolution
Horizontal and vertical re-trace
Refresh, refresh rate
Interlace
NTSC, PAL, S-video, Composite, Component
HDTV

26. LED Display Interactive Computer Graphics
Flat-panel displays: use different technologies
LED display: R, G, B LEDs for each pixel, 1k pixels: 3k subpixels
Pentile RGBG arrangement, 16 subpixels: 8G, 4R, 4B, one-third fewer subpixels, 1k pixels: 2k subpixels
Equal luminance resolution fewer pixels, less color resolution – matching human eye
Question. Why do we use 3 colors, R, G, and B? Why not 5?

27. Graphics Paradigms Modeling Rendering Photo-realistic: Interactive:
Interactive Computer Graphics
Graphics Paradigms
Modeling
Rendering
Photo-realistic:
Ray tracing
Radiosity
Interactive:
Projection – camera model
Transformations, clipping
Shading
Texture mapping
Rasterization

28. Interactive Computer Graphics
Ray Tracing
Ray Tracing

29. Ray-traced blob Interactive Computer Graphics
How would you write a program to generate this image?

30. How does Ray-Tracing work?
Interactive Computer Graphics
How does Ray-Tracing work?
Modeling
Build a 3D model of the world
Geometric primitives
Light sources
Material properties
Simulate the bouncing of light rays
Trace ray from eye through image pixel to see what it hits
From there, bounce ray in reflection direction, towards light source, etc.
Thus, model physics of emission, reflection, transmission, etc. (backwards)

31. Modeling the World Interactive Computer Graphics camera {
location <0, 5, -5>
look_at <0, 0, 0>
angle 58 }
light_source { <-20, 30, -25>
color red 0.6 green 0.6 blue 0.6 }
blob {
threshold 0.5
sphere { <-2, 0, 0>, 1, 2 }
cylinder { <-2, 0, 0>, <2, 0, 0>, 0.5, 1 }
cylinder { <0, 0, -2>, <0, 0, 2>, 0.5, 1 }
cylinder { <0, -2, 0>, <0, 2, 0>, 0.5, 1 }
pigment { color red 1 green 0 blue 0 }
finish { ambient 0.2 diffuse 0.8 phong 1 }
rotate <0, 20, 0>
To make a complex scene, you'd use a modeling program to generate this stuff, but let's do it by hand for this assignment. You can keep your scene simple.

32. Interactive Computer Graphics
Ray thru pixel

33. Flat blob Interactive Computer Graphics
Why does this image look so poor? It's entirely flat. There's no real sense of shape, of depth. Why is that?
Our previous model wasn't very accurate – realistically, more light bounces off the object toward our eye when the object is pointed directly toward a light; less when the surface slants away from light sources.

34. Bounce toward lights Interactive Computer Graphics
How would you know what color to use for each pixel of the image? The basic idea is to "shoot a ray" from the eye, through the pixel of the image, and see what part of the world model it hits first. If it hits a red object, we color that pixel red.
What would you get using this approach, do you think?

35. Interactive Computer Graphics
Shadows

36. Interactive Computer Graphics
Shaded blob

37. Interactive Computer Graphics
Blob with Highlights

38. Interactive Computer Graphics
Blob with ground plane

39. Blob with transparency
Interactive Computer Graphics
Blob with transparency

40. Interactive Computer Graphics
Blob with refraction

41. Interactive Computer Graphics
Types of illumination
Ambient – "light soup" that affects every point equally
Diffuse – shading that depends on the angle of the surface to the light source
Specular – 'highlights.' Falls off sharply away from the reflection direction
Example: lighted teapot

42. Interactive Computer Graphics
What are these made of?

43. Material types Dielectrics (non-conductors): Conductors (metals)
Interactive Computer Graphics
Material types
Dielectrics (non-conductors):
In body reflection, light penetrates the surface and is affected by material pigment
Highlights are the color of the light source
Examples: paint, plastic, wood, …
Conductors (metals)
No light penetrates the surface
Highlight and "body" reflection are affected equally by the material
Same color for diffuse and specular reflection

44. Interactive Computer Graphics
Finishes

45. Interactive Computer Graphics
Textures

46. Interactive Computer Graphics
Surface (Ripples)

47. Interactive Computer Graphics
POV-Ray Primitives

48. Constructive Solid Geometry
Interactive Computer Graphics
Constructive Solid Geometry

49. Interactive Computer Graphics
Sunsethf

50. How to ray-trace… Transparency? Refraction? Reflection? Fog?
Interactive Computer Graphics
How to ray-trace…
Transparency?
Refraction?
Reflection?
Fog?
Anti-aliasing?

51. Drawbacks of ray tracing?
Interactive Computer Graphics
Drawbacks of ray tracing?
Time: many rays are needed per pixel…
Up to 25 rays through each pixel
Each ray may bounce and split many times
Each ray tested for intersection with many objects
E.g. 1M pixels * 25 rays per pixel * 40 rays per ray tree * 1000 objects = 1 trillion object intersection tests…
Too slow for real time?
Shape limitations
Hard lighting
No soft shadows, inter-object diffusion, etc
Need to be able to compute intersection with shape and ray, bounce angles, etc.

52. POV-Ray An excellent, free ray tracer: POV-Ray
Interactive Computer Graphics
POV-Ray
An excellent, free ray tracer: POV-Ray
We'll use for a brief intro to ray tracing
Runs on PC, Unix, Mac, Beowolf clusters, …
Installed on the computers in the Unix lab
You may wish to install on your own computer
First "lab": make a ray-traced image of four different types of primitives, one each plastic, glass, metal, and mirrored, over checked floor

53. Beyond Ray Tracing Problems with ray tracing: hard shadows
Interactive Computer Graphics
Beyond Ray Tracing
Problems with ray tracing:
hard shadows
no color bleeding
slow
How else could you generate realistic-looking images – without the hard shadows and aliasing? Color bleed?

54. Interactive Computer Graphics
Radiosity in POV-Ray

55. Radiosity Treat each patch as reflector and emitter of light
Interactive Computer Graphics
Radiosity
Treat each patch as reflector and emitter of light
Each patch affects every other patch depending on distance, orientation, occlusion etc.
Let light "bounce around" for a few iterations to compute the amount of light reaching a patch

56. Interactive Computer Graphics
Radiosity image

57. Interactive Computer Graphics
Radiosity - table

58. Interactive Computer Graphics
Image: Wikipedia

59. Interactive Computer Graphics
Radiosity example

60. Interactive Computer Graphics
Source: ACM

61. Interactive Computer Graphics
Form factors
We need to know the percentage of the light leaving one patch that reaches another (form factor).
How to compute?

62. Hemicube algorithm Form factor computation:
Interactive Computer Graphics
Hemicube algorithm
Form factor computation:
Put a hemicube around patch reference point
Render an image in each of five directions
Count pixels…

63. Radiosity summary Radiosity gives wonderful soft shading
Interactive Computer Graphics
Radiosity summary
Radiosity gives wonderful soft shading
But even slower than ray tracing…
Can't do reflection, refraction, specular highlights with radiosity
Can combine ray tracing and radiosity for best of both worlds (and twice the time)

64. Interactive techniques
Interactive Computer Graphics
Interactive techniques
Ray tracing and radiosity are too slow
We'll concentrate on interactive techniques
What kind of rendering can be done quickly?

65. Shutterbug - Orthographic
Interactive Computer Graphics
Shutterbug - Orthographic
Mention source of images
Mention renderman, pixar

66. Interactive Computer Graphics
- Perspective
Perspective projection

67. Interactive Computer Graphics
- Depth Cueing
Depth Cueing

68. Interactive Computer Graphics
- Depth Clipping
Depth clipping

69. Interactive Computer Graphics
- Colored Edges
Colored edges

70. - Hidden line removal Interactive Computer Graphics
Hidden lines removed

71. - Hidden surface removal
Interactive Computer Graphics
- Hidden surface removal
Hidden surfaces removed
Ambient lighting only

72. - Flat shading Interactive Computer Graphics
Individually shaded polygons with diffuse reflection

73. - Gouraud shading Interactive Computer Graphics
Gouraud-shaded polygons with diffuse reflection

74. - Gouraud/specular Interactive Computer Graphics
Gouraud shading with specular reflection

75. - Gouraud/phong Interactive Computer Graphics
Phong shading with specular reflection

76. Interactive Computer Graphics
- Curved surfaces
Curved surfaces

77. - Improved illumination
Interactive Computer Graphics
- Improved illumination
Improved illumination model with multiple lights

78. Interactive Computer Graphics
- Texture mapping
Texture mapping

79. - Displacements, shadows
Interactive Computer Graphics
- Displacements, shadows
Displacement mapping, shadows

80. Interactive Computer Graphics
- Reflections
Reflections

81. Interactive Computer Graphics
Rendering pipeline
Where does lighting happen? Shading? Reflections? Resizing and moving objects? Textures? Bump mapping? Displacement mapping? Hidden surface removal?
This all happens in the “graphics pipeline”

82. History of Interactive Graphics
Interactive Computer Graphics
History of Interactive Graphics
Here's Intel's vision of computer graphics hardware history
Intel's Larrabee – many cores, fully programmable, can be used for graphics
(or other things)
Released Dec. 09 – hasn't really caught on.
Graphics is data parallel. (Same processing on each pixel – ideal for parallelization.) Except not as much as Intel thought. Many different kinds of parallelism to exploit.
Intel was also shocked to find out that it can take longer to implement something in software on multi-core CPUs than in custom hardware. The were slow to the plate.
Also, hardware is more efficient for performance and cost/energy
(This take draws from a paper by Andrew Richards, "Why Intel Larrabee really stumbled: developer analysis")

83. Nvidia: Moore's Law is Dead, Multi-core Not Future
Interactive Computer Graphics
Nvidia: Moore's Law is Dead, Multi-core Not Future
According to NVidia, the future is not a few cores optimized for serial performance speed, it's many smaller processors optimized for power consumption.