1. Class 11 LBSC 690 Information Technology Multimedia

2. Agenda Questions Images Audio Transmission Virtual reality Midterm structure Midterm review

3. Basic Image Coding Raster of picture elements (pixels) –Each pixel has a “color” Binary - black/white (1 bit) Grayscale (8 bits) Color (each pixel has three dots) –Red, green, blue Screen –A 1024x768 image requires 2.4 MB So a picture is worth 400,000 words!

4. Some Questions Use this to answer: –How many images can a 2 GB hard drive store? Nobody would use images! –How long does it take to send one by modem? Imagine how slowly web pages would load But real images don’t have these problems –How do we get around these problems?

5. Compression General goal: reduce redundancy –Send the same information using fewer bits more telephone calls in one cable more faxes per minute more images stored per disk better quality video images Two basic strategies: –Lossless, Lossy –The two strategies can be combined

6. Lossy Compression Example - Palette selection –No picture uses all 16 mission colors –Select a palette of 256 colors Can represent with 1byte instead of 3 Then look up each color in the palette JPEG (.jpg) –Standard lossy compression for images –Eliminates detail that’s not seen by humans –Uses frequency representation

7. Lossless Compression Run Length Encoding (RLE) –Pixels are organized into lines –Most pixels are the same as the one before That can be coded in 1 bit (1/24 the space) –Smaller files take less time to transmit GIF (.gif) –Standard lossless compression format

8. Moving Images One image frame is much like the next –An additional source of redundancy MPEG-1 (.mpg) can handle small screens –Compression requires extensive computation Special purpose hardware needed to run in real-time –Pentium processors can decode it MPEG-2 is needed for full-screen video –Not yet widely used

9. Audio Sample at twice the highest frequency –One or two bytes per sample –Voice (0-4 kHz) requires 8 kB/s –Music (0-22kHz) requires 44 kB/s Compression strategies –Lossy is pretty good for voice Only some of the frequencies are actually used –Lossless is better for music

10. Transmission MIME –Attach a standard format to message Formats include.gif,.jpg,.mpg, and.au Messages include email and web pages –The whole file is sent first, then played

11. Streaming Audio and Video Streaming protocols –Replay starts almost immediately –RealVideo has emerged as the standard Streaming video challenges –Sent in small packets –Sometimes arrive out of order –Compensate by storing some in a buffer Introduces a delay Modems carry audio better than video –Video data requires high “bandwidth” –Real Video compensates with lower frame rates

12. Virtual Reality Displays Visual –Position/motion, color/contrast, symbols Auditory –Position/motion, tones, speech Haptic –Mechanical, thermal, electrical, kinesthethic Olfactory –Smell, taste Vestibular

13. Types of Displays Image display –Fixed view, movable view, projection Acoustic display –Headphones, speakers, within-ear monitors Tactile display –vibrotactile, pneumatic Force feedback –dexterous handmaster, joystick, pen

14. More Display Types Inertial Display –Motion-based simulators Olfactory Display –Chemical (requires re-supply) Locomotive display –Stationary bicycle, treadmill,... Temperature Display

15. Wearable Visual Interfaces Head mounted displays –Two liquid crystal (camcorder-like) displays –Cables for video and position tracking Binocular omni-oriented monitor –Simpler and more comfortable –Limited range of motion Shuttered glasses –Used with fixed and projection displays

16. Some Input Techniques Arms and hands –Aimed movement, grasp, press, tap, sign/gesture, write/draw/paint/sculpt, crawl/swing, rub Legs –Walk/run/skip/hop/crawl, swim, aimed movement Face –Expression, speak, sing/hum, breathe Body –Posture, appearance/clothing

17. Input Devices Position tracking –Mechanical, magnetic, acoustic, GPS, inertial Gloves and suits –Fiber optic, flexible strip, exoskeleton Locomotive –Stationary bicycle, treadmill, stairs, shoes, rowing Conventional input devices (haptic, speech) Image processing (position, gesture) Facial sensing, eye tracking, biosignal

18. Position Tracking Magnetic –Three fixed emitters, three movable sensors Acoustic –Time delay systems have variable update rates –Phase coherent systems can jump lanes Differential GPS –Requires unobstructed sight lines to satellites Inertial –Large, short term, requires initial calibration

19. Exam Structure One hour and 15 minutes Approximately 4 questions –Each may have multiple parts Open Book (Oakman only) –You may hand write anything in your Oakman –No extra pages of notes The software you may use will be specified You may bring a calculator

20. Exam Advice The only goal is to get points! –Spend each minute in the best place Develop a strategy for each question type –Guessing CAN hurt on multiple choice –Don’t write a page when a sentence will do Study concepts, not details –Grading rewards conceptual understanding –Don’t expect a clone of the sample exams