Digital Audio Multimedia Systems (Module 1 Lesson 1) Summary: Basic concepts underlying sound Facts about human perception of sound Computer representation of sound (Audio) A brief introduction to MIDI Sources: My research notes Dr. Ze-Nian Li’s course material at: http://www.cs.sfu.ca/CourseCentral/365/li/ In this lesson we will study one of the media types that is the core of most Multimedia systems, audio. We will first understand the basic concepts underlying sound, We will also look at some facts about human perception of sound As most natural phenomena, sound occurs in an analog form which has to be digitized in order to represent it and store it on a computer. We will look at this process of conversion with the help of a fundamental theorem in digital signal processing called the “Nyquist Sampling Theorem”. Lastly, we will look at an alternative mechanism for representation of sound that has both good compression properties and provides ease of non-linear editing. The crux of the material is derived again from Dr. Ze-Nian Li’s course material at Simon Fraser University.

Sound Facts Sound is a continuous wave that travels through the air The wave is made up of pressure differences. Sound is detected by measuring the pressure level at a location Sound waves have normal wave properties (reflection, refraction, diffraction etc.) The human Ear detecting Sound Sound is a continuous wave that travels through air. The wave itself is comprised of pressure difference. Detection of sound is accomplished by measuring these pressure levels and their succession in time. The human ear does this detection naturally when the wave with its pressure differences impinges on the

Sound Facts Wave Characteristics Frequency: Represents the number of periods in a second and is measured in hertz (Hz) or cycles per second. Human hearing frequency range: 20Hz to 20kHz (audio) Amplitude: The measure of displacement of the air pressure wave from its mean. Related to but not the same as loudness Amplitude Air Pressure Time One Period One particular frequency component Like all waves sound can be characterized by the properties, frequency and amplitude. In order to understand these characteristics lets take a look at one frequency component of sound. It’s a constant frequency wave. Most sounds can be expressed mathematically as a combination of such waves of differing frequencies and amplitude. As is done using a Fourier series. The frequency refers to the rate at which the wave repeats. It is expressed as cycles per second or by the units hertz. The human hear is capable of perceiving wave frequencies in the range 20Hz and 20KHz, which is audio in nature. The amplitude is a measure of the displacement of the wave from the mean. For human perception this is related but not the same as loudness.

Principles of Digitization Why Digitize? Microphones, video cameras produce analog signals (continuous-valued voltages) To store audio or video data into a computer, we must digitize it by converting it into a stream of numbers. Sound as analog signal As I noted before sound is a continuous wave and as all natural phenomena is analog in nature. Microphones detect this analog input, which is nothing but a continuous sequence of voltages. However, in order to store this input in a computer one has to convert it to a digital form, that is into 0s and 1s. Further a continuous wave has infinite resolution which cannot be represented in a computer.

Principles of Digitization Sampling: Divide the horizontal axis (time) into discrete pieces Quantization: Divide the vertical axis (signal strength - voltage) into pieces. For example, 8-bit quantization divides the vertical axis into 256 levels. 16 bit gives you 65536 levels. Lower the quantization, lower the quality of the sound Linear vs. Non-Linear quantization: If the scale used for the vertical axis is linear we say its linear quantization; If its logarithmic then we call it non-linear (-law or A-law in Europe). The non-linear scale is used because small amplitude signals are more likely to occur than large amplitude signals, and they are less likely to mask any noise. Digitization is achieved by recording or sampling the continuous sound wave at discrete points. The more frequently one samples the closer one gets to capturing the continuity of the wave. Therefore, sampling is the process of dividing the horizontal (time-axis) into discrete points. The other aspect of digitization is the measurement of the voltages at these discrete sampling points. As it turns out these values may be of arbitrary precision, that is we could have values containing small fractions or decimal numbers that take more bits to represent. To cope with this arbitrary precision we use quantization which divides the vertical axis (signal strength or voltage) into discrete points. For example, 8-bit quantization divides the axis into 256 discrete voltage levels. While we are on the topic of quantization, it is important to note that not all quantization is uniform or linear. That is the vertical axis need not always be a linear scale, a non-linear logarithmic scale is used in mu-law encoding.

Sampling and Quantization 3-bit quantization Sampling rate: Number of samples per second (measured in Hz) E.g., CD standard audio uses a sampling rate of 44,100 Hz (44100 samples per second) 3-bit quantization gives 8 possible sample values E.g., CD standard audio uses 16-bit quantization giving 65536 values. Why Quantize? To Digitize! Here is a demonstration of the concepts of sampling and quantizaion. Again we consider a sinusoidal sound wave and sample it at discrete points. We notice that the resulting sampled values could be of arbitrary precision. Therefore we use a 3-bit quantization to discretize these values to finite values. One important observation to make is the process of quantization gives us discrete finite precision values which are needed for representation on a computer. Another important question to ask ourselves is: How often should we sample the signal so as achieve a faithful digital representation of the analog signal. This question will be answered by the Nyquist Sampling theorem which we will look at momentarily.

Nyquist Theorem Consider a sine wave Sampling once a cycle Appears as a constant signal Once again lets consider a sinusoidal sound wave. If we were to sample it once every cycle. It would appear as a constant signal. On the other hand lets say we sample it twice every three cycles or 1.5 times a cycle then it would appear low frequency sine wave with some extrapolation. As a matter of fact it may appear as a sawtooth wave without extrapolation. Nyquist sampling theorem suggests that in order to get lossless digitization that is a faithful digital representation of the analog signal one has to sample at twice the rate of the maximum frequency component. Thus in this example one would have to sample the wave two times each cycle. Sampling 1.5 times each cycle Appears as a low frequency sine signal For Lossless digitization, the sampling rate should be at least twice the maximum frequency responses

Application of Nyquist Theorem Nyquist theorem is used to calculate the optimum sampling rate in order to obtain good audio quality. The CD standard sampling rate of 44100 Hz means that the waveform is sampled 44100 times per sec. Digitally sampled audio has a bandwidth of (20 Hz - 20 KHz). By sampling at twice the maximum frequency (40 KHz) we could have achieved good audio quality. CD audio slightly exceeds this, resulting in an ability to represent a bandwidth of around 22050 Hz. Lets look at some practical applications of the theorem. Once again, Nyquist theorem is used to calculate the optimum sampling rate in order to obtain good sound quality. For CD audio the sampling rate is 44,100 Hz or 44,100 samples per second.

Quantization (Quality ->SNR) In any analog system, some of the voltage is what you want to measure (signal), and some of it is random fluctuations (noise). SNR: Signal to Noise ratio captures the quality of a signal (dB) Signal to Quantization Noise Ratio (SQNR) The quantization error (or quantization noise) is the difference between the actual value of the analog signal at the sampling time and the nearest quantization interval value. The largest (worst) quantization error is half of the interval? Lets now take a look at the quality of a digital signal. This is expressed by the signal-to-noise ratio and is given by the formula… As a demonstration of this concept, lets determine the quality of a signal which has noise introduced due to quantization. Recall that quantization results in approximating the voltage values to the nearest discrete point on the y axis. This process introduces error or noise. The quantization noise is the difference between the actual value of the analog signal at a sampling point and the nearest quantization value. Obviously in linear quantization, the worst case quantization error is half the size of the interval. V2signal V2noise Vsignal Vnoise SNR = 10 log = 20 log

SQNR Calculation (WC) If we use N bits per sample, the range of the digital signal is: -2N-1 to  2N-1 The worst-case signal to quantization noise ratio is given by: Vsignal Vquant - noise 2N-1 1/2 SQNR = 20 log = 20 log = N x 20 log 2 = 6.02N (dB) Lets calculate the worst case signal-to-quantization ratio. Assume we are using N bits per sample. Therefore the range of quantization values is from –2 to the N-1 to 2 to the N-1. The SQNR is given by 20 log the ratio of the maximum signal value by the maximum qunatization noise. That is 2 to the N-1 divided by half of 2 to the –1. Which gives us log 2 to the N. This is approximately 6.02N. We note that by adding another bit we can add approximately 6 dB to the worst-case signal quality. So 16 bits enable a maximum SQNR of 96 dB. Each bit adds about 6 dB of resolution, so 16 bits enable a maximum SQNR = 96 dB.

Miscellaneous Audio Facts Typical Audio Formats Popular audio file formats include .au (Unix), .aiff (MAC, SGI), .wav (PC, DEC) A simple and widely used audio compression method is Adaptive Delta Pulse Code Modulation (ADPCM). Based on past samples, it predicts the next sample and encodes the difference between the actual value and the predicted value. Before we wrap up the discussion of digital audio it is interesting to note a few miscellaneous facts about digital audio. First we find that the most popular audio file formats are .au in UNIX, .aiff on the Mac and .wav on the PC. Note that these are uncompressed formats. Formats like mp3 and real-audio’s ram involve compression which will be covered later. While we are talking about compression. I would like to draw your attention to a simple but popular method for compression called Adaptive Delta PCM or ADPCM. In t his method, the samples are predicted based on the behavior of past samples and the difference between the predicted value and the actual observed value is coded. To understand the source of the compression note that the difference (if the prediction is sound) is a smaller number that the actual value and encoding it will take fewer bits.

Audio Quality vs. Data Rate Sample Rate (kHz) Bits per Sample Mono/ Stereo Data Rate (kBytes/sec) (uncompressed) Frequency Band Telephone 8 Mono 8  200-3400 Hz AM Radio 11.025 11.0  540-1700 KHz FM Radio 22.050 16 88.2 CD 44.1 176.4 20-20000 Hz DAT 48 192.0 CD coding of Audio when done without compression takes 176 kilobytes of space for 1 second of recorded audio.

Synthesizer/Keyboard MIDI Interface/Sound Card Musical Instrument Digital Interface a protocol that enables computer, synthesizers, keyboards, and other musical devices to communicate with each other. Setup: MIDI OUT of synthesizer is connected to MIDI IN of sequencer. MIDI OUT of sequencer is connected to MIDI IN of synthesizer and "through" to each of the additional sound modules. Working: During recording, the keyboard-equipped synthesizer is used to send MIDI message to the sequencer, which records them. During play back, messages are sent out from the sequencer to the sound modules and the synthesizer which will play back the music. THRU IN OUT IN OUT Synthesizer/Keyboard MIDI Interface/Sound Card (Sequencer) IN THRU MIDI Module A IN THRU MIDI Module B Etc. Typical Sequencer setup

MIDI: Data Format Information traveling through the hardware is encoded in MIDI data format. The encoding includes note information like beginning of note, frequency and sound volume; upto 128 notes The MIDI data format is digital The data are grouped into MIDI messages Each MIDI message communicates one musical event between machines. An event might be pressing keys, moving slider controls, setting switches and adjusting foot pedals. 10 mins of music encoded in MIDI data format is about 200 Kbytes of data. (compare against CD-audio!)