1. Continuous Media: Audio, Animation, Video
Multimedia
Continuous Media: Audio, Animation, Video
T.Sharon-A.Frank

2. Sound Conversion of energy into vibrations in the air.
Most sound sources vibrate in complex ways leading to sounds with components at several different frequencies.
Frequency spectrum – relative amplitudes of the frequency components.
Range of human hearing: roughly 20Hz–20KHz, falling off with age.

3. Provide personal contact Tell a story
Use of Audio
Convey Information
Set a mood
Capture attention
Explain a process
Provide personal contact
Tell a story
T.Sharon-A.Frank

4. Nature of Sound Sound is produced by the vibration of matter.
Pressure variations are created in air surrounding matter.
Pattern of oscillation is called a waveform.
T.Sharon-A.Frank

5. Waveforms Sounds change over time
e.g. musical note has attack and decay, speech changes constantly.
Frequency spectrum alters as sound changes.
Waveform is a plot of amplitude against time:
Provides a graphical view of characteristics of a changing sound.
Can identify syllables of speech, rhythm of music, quiet and loud passages, etc.

6. Computer Representation of Sound:
Basic Sound Concepts
Waveform:
Period
Frequency
Amplitude
Computer Representation of Sound:
Sampling
Quantization
T.Sharon-A.Frank

7. Period
Repeating portion of the sound wave, although sound waves are never perfectly smooth or uniformly periodic.
T.Sharon-A.Frank

8. Frequency
Reciprocal of the period, that is, number of periods in a second; measured in hertz (Hz) or cycles per second.
Human hearing frequency range is from 20Hz to 20KHz:
Infrasound: Hz
Ultrasound: 20KHz - 1GHz
Hypersound: 1GHz - 10THz
T.Sharon-A.Frank

9. Amplitude
Measure of displacement of the air pressure wave from its mean.
Subjectively heard as loudness.
amplitude
T.Sharon-A.Frank

10. Sound Wave Example
T.Sharon-A.Frank

11. Idea of Audio Digitization
Smooth continuous curve of waveform is not directly represented in computer.
Amplitude of waveform is measured and recorded at regular time intervals - process is called encoding.
Each interval is called a sample.
Need Analog-to-Digital Converter (ADC).
T.Sharon-A.Frank

12. Audio Digitization Converting signal from analogue to digital form
Analogue signal can vary continuously; digital is restricted to discrete values.
Two-stage process:
Sampling – measure the value at discrete intervals.
Quantization – restrict the value to a fixed set of quantization levels.

13. Digital Sampling
Sample Height
Samples
Time
Samples
T.Sharon-A.Frank

14. Example: 3 bit quantization
Digital Quantization
Sample Height
Example: 3 bit quantization
Samples
Samples
T.Sharon-A.Frank

15. Audio Encoding
(a) A sine wave. (b) Sampling the sine wave. (c) Quantizing the samples to 4 bits.
T.Sharon-A.Frank

16. Process of Sampling Audio
T.Sharon-A.Frank

17. Number of samples per second measured in hertz (Hz).
Sampling Rate
Number of samples per second measured in hertz (Hz).
More samples = higher quality = bigger files.
Example:
CD quality = 44,100 Hz (44.1 KHz)
Others: 8,000 Hz, 11,025 Hz, 22,050 Hz
T.Sharon-A.Frank

18. Sampling Size Number of bits used to represent each sample.
Called also Bit Depth and Sound Resolution.
8-bit - telephone quality (typical of Web).
16-bit - CD quality.
Examples:
Web quality ( KHz, 8bit)
16 bit, 44.1kHz
8 bit , 44.1KHz
T.Sharon-A.Frank

19. Channels
Audio files can support from 1 to 6 channels of audio information.
Most common:
mono (1 channel)
stereo (2 channels)
4-6 channels are usually used for surround sound.
T.Sharon-A.Frank

20. Record Settings Menu for Audio
T.Sharon-A.Frank

21. Sample Audio File Formats
aif/aiff
wav au
mov
MPEG/MP3 (MPEG 1 Level 3)
RA/RAM
MIDI (Musical Instrument Digital Interface)
T.Sharon-A.Frank

22. Audio File Formats (platform specific)
aif (aiff)
Audio Interchange File Format
original audio format for Mac
supported by Windows and other platforms
wav
Waveform Audio File Format
original Windows audio file format
supported on Mac au
u-law (myoo-law)
Unix standard audio format
T.Sharon-A.Frank

23. Audio File Formats (container formats)
mov (qt)
QuickTime, not just Apple/MAC
mpg (mpe, mpeg)
Moving Pictures Expert Group
High quality, excellent compression file format
Files tend to be small and sound good
ra (ram)
RealAudio
T.Sharon-A.Frank

24. Real Audio Format First example of streaming audio on the Web.
Quality like that of an AM radio station.
Best used for narration and not for music or other sounds.
Must have RealPlayer on your system to listen to files.
T.Sharon-A.Frank

25. MIDI MIDI = Musical Instruments Digital Interface
Instructions about how to produce music, which can be interpreted by suitable hardware and/or software.
Standard protocol for communicating between electronic instruments (synthesizers, samplers, drum machines).
Allows instruments to be controlled by hardware or software sequencers.

26. MIDI and Computers
MIDI interface allows a computer to send MIDI data to instruments.
Store MIDI sequences in files, exchange them between computers, incorporate them into multimedia.
Computer can synthesize sounds on a sound card, or play back samples from disk in response to MIDI instructions
Computer becomes primitive musical instrument (quality of sound inferior to dedicated instruments).

27. Uncompressed Audio Digital Telephony
Sampled at 8KHz (8K cycles per second)
Quantized to 8 bits per sample (1 Byte)
=> 64,000bits/s
Bandwidth = 64,000/1,024 =
Total: 62.5Kb/s
T.Sharon-A.Frank

28. Uncompressed Stereo Audio CD Quality
Sampled at 44.1KHz (44.1K cycles per second)
Quantized to 16 bits per sample (2 bytes)
Two channels for stereo
Bandwidth =
2 * 44,100 * 16 = 1,411,200b/s
1,411,200/1024= 1,378Kb/s
Total: ~1.4Mb/s
T.Sharon-A.Frank

29. Animation The creation of moving pictures one frame at a time:
Literally 'to bring to life‘.
e.g. make a sequence of drawings on paper, in which a character's position changes slightly in each drawing.
Photograph drawings in sequence, using movie camera that advances one frame at a time.
Play back resulting film – character will move.

30. Use of Animation
Capture the viewer’s attention, entertain, or promote.
Explain a system or process.
Set a mood or create an on-screen environment.
T.Sharon-A.Frank

31. Animation Principles Persistence of vision:
object seen by human eye remains mapped on retina for a brief time after viewing.
display series of images rapidly and they blend together to create illusion of movement.
requires at least 16 frames per second (fps) to look seamless.
Television: displayed at 30 frames per second.
Movies: displayed at 48 frames per second.
T.Sharon-A.Frank

32. Cell Animation (Frame Animation)
Start with first and last image in motion (called keyframes).
Draw images between keyframes (process called tweening/morphing) - small changes between images.
Images are then layered onto background scene.
Images displayed rapidly
15/24/30 frames per second.
T.Sharon-A.Frank

33. Captured Animation
Logic and procedural concepts are same as in cell animation (keyframe, tweening, layering techniques).
But use computer and video camera.
Frame grabbing – record each frame to disk.
Edit non-linearly like for video.
Can also use scanner or digital still camera, or create each frame in a graphics program (e.g., Painter).

34. Cell-based Techniques
Morphing:
uses frames to create illusion of one object changing into another
more keyframes = smoother morph.
Need at least 15 frames per second.
Professional quality uses frames per second.
Cell-based animation is done more easily on computer than by hand.
Easier to create and process than movies.
T.Sharon-A.Frank

35. Path-based Animation (Vector Animation)
Create animated objects by following object’s transition over a line or vector.
Artist creates one drawing and a path.
Computer program manipulates object by drawing frames as object travels over the path.
T.Sharon-A.Frank

36. Sample Animation File Formats
Animated GIF
mov (QuickTime)
SWF (Flash)
PNG (Portable Network Graphics)
T.Sharon-A.Frank

37. Animated GIFs
Sequence of images can be stored in a single GIF file, and displayed one after another by a Web browser or other software:
No browser plug-in required
Can specify looping, delay between frames
256 colour palette
No sound.

38. SWF Popular Web animation format.
Usually generated by Macromedia Flash.
Vector animation format
Motion represented as numerical operations on vector data.
Can also include bitmapped images (e.g., as backgrounds).

39. Video is sequence of images displayed at constant rate.
Rate is in frames per second (fps).
Needs around fps for smooth motion.
For example, video is displayed at fps.
T.Sharon-A.Frank

40. Explain a complex process Personalize a Multimedia experience
Use of Video
Tell a story
Entertain an audience
Explain a complex process
Personalize a Multimedia experience
T.Sharon-A.Frank

41. Movie - Characteristics
Built from Frames - Each frame is an Image
Image characteristics (i.e., resolution, colors)
Clip
Scene
Shot
Frame
Frame
T.Sharon-A.Frank

42. A movie may consist of several files
T.Sharon-A.Frank

43. Sample Video File Formats
MOV
AVI
RA/RAM
ASF
WMV
MPEG1/2/4
FLV
T.Sharon-A.Frank

44. MPEG MPEG = Moving Picture Experts Group.
Family of compression algorithms.
High compression rate.
Algorithm stores only changes from one frame to another rather than each entire frame.
Video info then encoded using DCT.
Lossy compression but loss of data is generally imperceptible to human eye.
T.Sharon-A.Frank

45. MPEG Video Format MPEG-1: used for video CD 352x240 @30 frames/second.
ISO/IEC MPEG (Moving Picture Experts Group) Series of standards including:
MPEG-1: used for video CD 352x240 @30 frames/second.
MPEG-2: used for compressing DVD and high-definition television (HDTV).
MPEG-4: used to transmit audio, video, and graphics at low bitrate – can be delivered over very slow connections (56 Kbps).
T.Sharon-A.Frank

46. Video Acquisition TV set and PC monitor look similar. TV is analog
Gets info from continuously varying broadcasting waves.
PC is digital
Data displayed is digital.
Digitization performed by video capture card.
Typical card is combination of hardware and software.
Capture card typically creates uncompressed files that then must be passed into a codec chip or software.
T.Sharon-A.Frank

47. Issues Affecting Video File Size
Movie Length - simple principle: shorter video clips result in smaller files.
Frame Size - full-screen video is 640x480. Typical frame size on WWW = 160x120 although can handle 320x240.
Frame Rate:
TV quality = 30 fps
Web = 15 (or even 10) fps.
T.Sharon-A.Frank

48. TV Computer Display Video Presentation PAL NTSC SECAM HDTV
T.Sharon-A.Frank

49. TV Standards NTSC (National Television Standard Committee)
Used by TV and video industries in US and Japan Taiwan, parts of South America, …
PAL (Phase Alternating Line)
Used in Western Europe, Australia & New Zealand, China , …
SECAM (Sequential Color and Memory)
France and former Soviet Union
Standard only used for transmission
Very different technology than PAL.

50. Video (PAL, HDTV) Bandwidth
PAL Resolution: 640 x 480
3 bytes per pixel
Frame rate: 25 fps
Bandwidth:
640 * 480 * 25 * 24 = 184,320,000b/s
Total: ~176Mb/s
HDTV bandwidth
increased by factor
of 5.33: ~936Mb/s
T.Sharon-A.Frank

51. More PAL and NTSC Examples
PAL uncompressed
768x576 pixels per frame
x 3 bytes per pixel
x 25 frames per second
≈ 31 MB per second
≈ 1.85 GB per minute
NTSC uncompressed
640x480 pixels per frame
x 3 bytes per pixel
x 30 frames per second
≈ 26 MB per second
≈ 1.6 GB per minute