IT 2002 – Graphics & Multimedia www.hndit.com IT 2002 – Graphics & Multimedia Week 7- Audio ATI Kurunegala

What is Audio ? Audio is a key component of the communication. www.hndit.com What is Audio ? Audio is a key component of the communication. Presence of sound greatly enhances the effect of mostly graphics presentation. Especially in a video or with animation Sound is the brain’s interpretation of electrical impulses being sent by inner ear through the nervous system In multimedia projects-you can use sound in different ways.

Audio Sound is a continuous wave that travels through the air. www.hndit.com Audio Sound is a continuous wave that travels through the air. The wave is made up of pressure differences. 11/12/2018

How do we hear sound? www.hndit.com Sound is detected by measuring the pressure level at a point When an acoustic signal reaches the otter- ear (Pinna), the generated wave will be transformed into energy and filtered through the middle-ear. The inner-ear (Cochlea) transforms the energy into nerve activity. In similar way, when an acoustic wave strikes a microphone, the microphone generates an electrical signal, representing the sound amplitude as a function of time. 11/12/2018

How does the ear work? www.hndit.com As the sound waves enter the ear, the ear canal increases the loudness of those pitches that make it easier to understand speech and protects the eardrum - a flexible, circular membrane which vibrates when touched by sound waves. The sound vibrations continue their journey into the middle ear, which contains three tiny bones called the ossicles, which are also known as the hammer, anvil and stirrup. These bones form the bridge from the eardrum into the inner ear. They increase and amplify the sound vibrations even more, before safely transmitting them on to the inner ear via the oval window. The Inner Ear (cochlea), houses a system of tubes filled with a watery fluid. As the sound waves pass through the oval window the fluid begins to move, setting tiny hair cells in motion. In turn, these hairs transform the vibrations into electrical impulses that travel along the auditory nerve to the brain itself.

Properties of sound wave www.hndit.com Properties of sound wave

www.hndit.com Properties of sound Period- the difference between the formation of two crests(peak) is termed as period Frequency –the number of peaks that occur in one seconds(1/period) Human hearing frequency range: 20 Hz - 20 kHz (audio), voice is about 500 Hz to 2 kHz. Wavelength – is the distance from the midpoint of one crest to the midpoint of the next crest

www.hndit.com Properties of sound Amplitude of a sound is the measure of displacement of the air pressure wave from its mean Bandwidth –is defined as difference between the highest and lowest frequency contained in a signal

Analog to Digital Recording Chain www.hndit.com Analog to Digital Recording Chain ADC Microphone converts acoustic to electrical energy. It’s a transducer. Continuously varying electrical energy is an analog of the sound pressure wave. ADC (Analog to Digital Converter) converts analog to digital electrical signal. Digital signal transmits binary numbers. DAC (Digital to Analog Converter) converts digital signal in computer to analog for your headphones.

Computer Representation of Audio www.hndit.com Computer Representation of Audio Speech is analog in nature and it is converted to digital form by an analog-to-digital converter (ADC). A transducer converts pressure to voltage levels. Convert analog signal into a digital stream by discrete sampling Discretization both in time and amplitude (quantization) 11/12/2018

Audio Encoding www.hndit.com electrical voltage input Audio Waves Converted to Digital electrical voltage input sample voltage levels at intervals to get a vector of values: (0, 0.2, 0.5, 1.1, 1.5, 2.3, 2.5, 3.1, 3.0, 2.4,...) A computer measures the amplitude of the waveform at regular time intervals to produce a series of numbers (samples). The ADC process is governed by various factors such as sample rate and quantization: binary number as output 11/12/2018

www.hndit.com Audio Encoding Sampling Rate: rate at which a continuous wave is sampled (measured in Hertz) Examples: CD standard - 44100 Hz, Telephone quality - 8000 Hz The audio industry uses 5.0125 kHz, 11.025 kHz, 22.05 kHz, and 44.1 kHz as the standard sampling frequencies. These frequencies are supported by most sound cards. How often do you need to sample a signal to avoid losing information? 11/12/2018

Audio Encoding Nyquist Theorem: www.hndit.com Audio Encoding Nyquist Theorem: Sampling rate must be at least twice as high as the highest frequency you want to represent. Capturing just the crest and trough of a sine wave will represent the wave exactly.

Audio Encoding Nyquist Sampling Theorem: www.hndit.com Audio Encoding Nyquist Sampling Theorem: If a signal f(t) is sampled at regular intervals of time and at a rate higher than twice the highest significant signal frequency, then the samples contain all the information of the original signal. Example: CD's actual sampling frequency - 22050 Hz, Due to Nyquist's Theorem - sampling frequency is 44100Hz. It depends on how fast the signal is changing,. Real Answer: twice per cycle (this follows from Nyquist sampling theorem) 11/12/2018

Nyquist’s sampling Theorem www.hndit.com A signal s(t) can be reconstructed from its samples, provided that it is sampled at least at a rate of twice of its maximum frequency. Maximum frequency of the Analog signal = fm Hz Sampling rate = 2fm samples /second Analog samples are called PAM (Pulse Amplitude Modulated) samples Amplitude Time Sampling time = 1/2fm seconds Voice band width – 4000 Hz (with a guard band 900 Hz) Sample rate required to reconstruct the signal form its samples is 8000 samples / sec. Color TV - BW = 4.6 MHz

Aliasing What happens if sampling rate not high enough? www.hndit.com Aliasing What happens if sampling rate not high enough? A high frequency signal sampled at too low a rate looks like … … a lower frequency signal. That’s called aliasing or foldover. An ADC has a low-pass anti-aliasing filter to prevent this. Synthesis software can cause aliasing.

Common Sampling Rates Sampling Rate Uses 44.1 kHz (44100) CD, DAT www.hndit.com Common Sampling Rates Which rates can represent the range of frequencies audible by (fresh) ears? Sampling Rate Uses 44.1 kHz (44100) CD, DAT 48 kHz (48000) DAT, DV, DVD-Video 96 kHz (96000) DVD-Audio 22.05 kHz (22050) Old samplers Most software can handle all these rates.

www.hndit.com Digitization Process of converting analog data into digital data is known as digitization. Ea Analog to Digital Conversion Ed Representing with a minimum loss of information Contains an infinite number of amplitudes Sampling - (PAM) Measuring the amplitude of the analog signal at regular intervals

www.hndit.com Audio Encoding The best-known technique for voice digitization is Pulse-Code Modulation (PCM). Voice 4000 Hz What is the PCM sampling rate? PCM provides analog samples which must be converted to digital representation. Each of these analog samples must be assigned a binary code. Each sample is approximated by being quantized as explained next. If voice data are limited to 4000 Hz, then PCM samples 8000 samples/second which is sufficient for the input voice signal. 11/12/2018

Pulse-Code Modulation (PCM). www.hndit.com Pulse-Code Modulation (PCM).

Three different sampling methods for PCM www.hndit.com Three different sampling methods for PCM

Pulse-Code Modulation www.hndit.com Encoding Quantizing Done in three steps Sampling (PAM) Quantizing and Linear Encoding Binary Stream

www.hndit.com Audio Encoding Quantization (sample precision): the resolution of a sample value. Samples are typically stored as raw numbers (linear PCM format) or as logarithms (u-law or A-law) Quantization depends on the number of bits used measuring the height of the waveform Example: 16-bit CD quality quantization results in over 65536 values 11/12/2018

3-bit Quantization 7 6 5 4 Amplitude 3 2 1 www.hndit.com 3-bit Quantization A 3-bit binary (base 2) number has 23 = 8 values. 1 2 3 4 5 6 7 Amplitude Time — measure amp. at each tick of sample clock A rough approximation

4-bit Quantization 14 12 10 8 Amplitude 6 4 2 www.hndit.com 4-bit Quantization A 4-bit binary number has 24 = 16 values. 2 4 6 8 10 12 14 Amplitude Time — measure amp. at each tick of sample clock A better approximation

The Digital Audio Stream www.hndit.com The Digital Audio Stream It’s just a series of sample numbers, to be interpreted as instantaneous amplitudes: one for every tick of the sample clock. Previous example: 11 13 15 13 10 9 6 1 4 9 15 11 13 9 This is what appears in a sound file, along with a header that indicates the sampling rate, bit depth and other things.

Common Sampling Resolutions www.hndit.com Common Sampling Resolutions Word length Uses 8-bit integer Low-res web audio 16-bit integer CD, DAT, DV, sound files 24-bit integer DVD-Video, DVD-Audio 32-bit floating point Software (usually only for internal representation)

16-bit Sample Word Length www.hndit.com 16-bit Sample Word Length A 16-bit integer can represent 216, or 65,536, values (amplitude points). We typically use signed 16-bit integers, and center the 65,536 values around 0. 32,767 -32,768

Audio File Size CD characteristics… - Sampling rate: www.hndit.com Audio File Size CD characteristics… - Sampling rate: 44,100 samples per second (44.1 kHz) - Sample word length: 16 bits (i.e., 2 bytes) per sample - Number of channels: 2 (stereo) How big is a 5-minute CD-quality sound file?

Audio File Size How big is a 5-minute CD-quality sound file? www.hndit.com Audio File Size How big is a 5-minute CD-quality sound file? 44,100 samples * 2 bytes per sample * 2 channels = 176,400 bytes per second 5 minutes * 60 seconds per minute = 300 seconds 300 seconds * 176,400 bytes per second = 52,920,000 bytes = 50.5 megabytes (MB)

DAC(Digital to Audio converter) Sample and Hold www.hndit.com DAC(Digital to Audio converter) Sample and Hold To reconstruct analog signal, hold each sample value for one clock tick; convert it to steady voltage. 1 2 3 4 5 6 7 Amplitude Time

www.hndit.com DAC: Smoothing Filter Apply an analog low-pass filter to the output of the sample-and-hold unit: averages “stair steps” into a smooth curve. 1 2 3 4 5 6 7 Amplitude Time

www.hndit.com Audio Formats Audio Formats are described by the sample rate and quantization Voice quality: 8-bit quantization, 8000 Hz u-law mono (8kBytes/s) 22 kHz 8-bit linear mono (22 kBytes/second) and stereo (44 kBytes/s) CD quality 16-bit quantization, 44100 Hz linear stereo (176.4 kBytes/s = 44100 samples x 16 bits/sample x 2 (two channels)/8000) 11/12/2018

Audio Formats Available formats on SUN au - Sun File Format www.hndit.com Audio Formats Available formats on SUN au - Sun File Format wav - Microsoft RIFF/waveform Format al - Raw A-law Data Format u - Raw u-law Data Format snd - NeXT File Format Available formats on Microsoft-Windows-based systems ( RIFF formats): Waveform audio file format for digital audio hardware MIDI file format for standard MIDI files Audio Video Interleaved (AVI) Indeo file format 11/12/2018

www.hndit.com Audio Formats RIFF (Resource Interchange File Format) forms the basis of a number of file formats. RIFF (similarly to TIFF - Tagged Image File Format) is a tagged file format. Tags allow applications capable of reading RIFF files to read RIFF files by another application, hence the word interchange in RIFF. Other Formats/Players - RealPlayer 7 (Windows NT) with RealAudio, MP3 (MPEG Audio Layer 3) audio, Midi players; MP3 players (MP3.com) 11/12/2018

Use of Audio in Multimedia www.hndit.com Use of Audio in Multimedia Can use sound in multimedia projects in two ways Content Sound provides information to audiences.(dialogs in movies or theater) Ambient sound consists of array of background & sound effects

Content Sound used in multimedia are : www.hndit.com Content Sound used in multimedia are : Narration : provides information about an animation that is playing on the screen Testimonials : these could be auditory or video sound tracks used in presentations or movies Voice-overs : these are sound for short instructions ,for example, to navigate the multimedia application Music –music may be used to communicate(as a song)

Ambient Sound used in multimedia www.hndit.com Ambient Sound used in multimedia Message reinforcement : the background sound you hear in real life, such as crowds at a ball game, can be used to reinforce the message that you wish to communicate. Back ground music : set the mood for audience to receive and process information by starting and ending a presentation with music. Sound effect : sound effects are used in presentation to liven up the mood and add effects to your presentations, such as sound attached to bulleted lists.

www.hndit.com Audio File formats

www.hndit.com Audio File Formats

www.hndit.com Audio File Formats

www.hndit.com Project Find all the sound files on your computer: WAV files (.wav), MIDI (.mid or .rmi), and any others (like .mp3 files) that you are able to download from the Web. Listen to several files of each format carefully, check their properties (right- clicks properties > advanced), and compare their quality with the properties like the size of file, time of the clip, compression used if any, sampling rate, etc. Make a table and record your results in the table. Mention whether these sounds are used (or can be used) as content audio or ambient audio, and explain in one sentence the message they give (if any).