1. CGMB324: Multimedia System Design
Chapter 8: Audio Compression

2. Objectives Upon completing this chapter, you should be able to:
understand some of the common audio compression methods
understand the basic concept of MPEG audio compression
apply audio compression in a multimedia system

3. Simple Audio Compression Methods
Motivation:
Traditional lossless compression methods (Huffman, LZW, etc.) usually don't work well on audio compression

4. Simple Audio Compression Methods
Existing Lossy methods:
Silence Compression - detect the "silence", similar to run-length coding
Apple has proprietary scheme called ACE / MACE. Lossy scheme that tries to predict where wave will go in next sample. About 2:1 compression.

5. Simple Audio Compression Methods
Linear Predictive Coding (LPC)  fits signal to speech model and then transmits parameters of model
 sounds like a computer talking, 2.4 kbits/sec.
Code Excited Linear Predictor (CELP)  does LPC, but also transmits error term
 audio conferencing quality at 4.8 kbits/sec.

6. Simple Audio Compression Methods
Adaptive Differential Pulse Code Modulation (ADPCM)
e.g., 16 or 32 Kbits/sec.
Encodes the difference between two or more consecutive signals;
The difference is then quantized (approximated) hence the loss
Adapts at quantization so fewer bits are used when the value is smaller.

7. Psychoacoustics Human hearing and voice
Frequency range is about 20 Hz to 20 KHz, most sensitive at 2 to 4 KHz.
Dynamic range (quietest to loudest) is about 96 dB
Normal voice range is about 500 Hz to 2 KHz
Low frequencies are vowels (louder than consonants, but low frequency) and bass
High frequencies are consonants

8. MPEG Audio Compression
Some facts
VCD: Max 1.5 Mbps.
Usually Mbps for audio and video.
About 1.15 Mbps for video and Mbps (224 Kbps) for audio.
The audio here is MPEG-1 Layer 2 audio (*.mp2).
This is for the VCD Standard.
Uncompressed CD audio is 44,100 samples/sec, 16 bits/sample, 2 channels > 1.4 Mbits/sec
Compression factor ranging from 2.7 to 24.

9. MPEG Audio Compression
With a compression rate of 6:1 (16 bit stereo sampled at 48 KHz is reduced to 256 kbits/sec) and optimal listening conditions, expert listeners could not distinguish between coded and original audio clips.
MPEG audio supports sampling frequencies of 32, and 48 KHz.
Newer technology like mp3, supports sampling frequencies as low as 11 KHz but is usually restricted to 16 bit and not 8 bits.

10. MPEG Audio Compression
Supports one or two audio channels in one of the four modes:
Monophonic -- single audio channel
Dual-monophonic -- two independent channels, e.g., English and French
Stereo -- for stereo channels that share bits, but not using Joint-stereo coding
Joint-stereo -- the encoder encodes to a mid channel, which is just one full channel by itself that is averaged out from the two original channels, and a side channel which contains all of the stereo separation. Instead of encoding two full channels like with normal stereo, the encoder only has to encode one channel and part of another. Sometimes there is too much stereo separation for JS to reproduce, if such a situation occurs, the encoder will switch back to normal stereo. This doesn't happen very often though, on most songs, JS is used on about 95% of the frames

11. MPEG Audio Compression
Steps in algorithm:
Use convolution filters to divide the audio signal (e.g., 48 KHz sound) into 32 frequency subbands --> subband filtering.
Determine amount of masking for each band caused by nearby band using a psychoacoustic model.
The psychoacoustic model is based on many studies of human perception. These studies have shown that the average human does not hear all frequencies the same. Effects due to different sounds in the environment and limitations of the human sensory system lead to facts that can be used to cut out unnecessary data in an audio signal
Masking is the phenomenon where a strong signal "covers" the sound of another signal such that the softer one cannot be heard by the human ear.

12. MPEG Audio Compression
For example, a jet engine noise can drown out music easily
If the power in a band is below the masking threshold, don't encode it.
Otherwise, determine number of bits needed to represent the coefficient (constant number)
Format the bitstream

13. MPEG Audio Compression

14. MPEG Audio Compression
MPEG Layers
MPEG defines 3 layers for audio. Basic model is the same, but codec complexity increases with each layer.
Divides data into frames, each of them contains 384 samples, 12 samples from each of the 32 filtered subbands as shown in Figure 1.

15. Figure 1

16. MPEG Audio Compression
Layer 1: DCT type filter with one frame and equal frequency spread per band. Psychoacoustic model only uses frequency masking.
the simplest and is best suited for bit rates above 128 kbits/sec per channel. For example, Philips' Digital Compact Cassette (DCC)[5] uses Layer I compression at 192 kbits/s per channel.

17. MPEG Audio Compression
Layer 2: Use three frames in filter (before, current, next, a total of 1152 samples). This models a little bit of the temporal masking. Temporal masking is the characteristic of the auditory system where sounds are hidden due to maskers before or even after that time. The effect of masking after a strong sound is called post-masking, and can be in effect up to 200 ms. The pre-masking, where a sound actually is masked by something which appears after it, is relatively short and may last up to 20 ms.
Also, temporal masking is a defence mechanism of the ear that is activated to protect its delicate structures from loud sounds. When exposed to a loud sound, the human ear will contract slightly, temporarily reducing the perceived volume of sounds that follow.

18. MPEG Audio Compression
This reflex manoeuvre, sometimes called "blinking" (by analogy to the eye), is meant to protect the delicate structures of the ear from potentially damaging sonic power. However, it also means that relatively loud sounds in an audio signal, such as a loud trumpet's note, will tend to overpower other sounds that occur just before and just after it, a phenomenon known as temporal masking.
has an intermediate complexity and is targeted for bit rates around 128 kbits/s per channel. Possible applications for this layer include the coding of audio for Digital Audio Broadcasting (DAB®)[6] , for the storage of synchronized video-and-audio sequences on CD-ROM, and the full motion extension of CD-interactive, Video CD.

19. MPEG Audio Compression
Layer 3 (MP3): Better critical band filter is used (non-equal frequencies), psychoacoustic model includes temporal masking effects (hiding), takes into account stereo redundancy, and uses Huffman coder.
the most complex but offers the best audio quality, particularly for bit rates around 64 kbits/s per channel. This layer is well suited for audio transmission over ISDN (Integrated Services Digital Network).

20. MPEG Audio Compression
Stereo Redundancy Coding:
Intensity stereo coding -- at upper-frequency subbands, encode summed signals (together) instead of independent signals from left and right channels.
achieves a saving in bitrate by replacing the left and the right signal by a single representing signal plus directional information
is psychoacoustically justified in the higher frequency range since the human auditory system is insensitive to the signal phase at frequencies above approximately 2kHz
a lossy coding method, primarily useful at low bitrates
Middle/Side (MS) stereo coding -- encode middle (sum of left and right) and side (difference of left and right) channels.
whenever a signal is concentrated in the middle of the stereo image, MS stereo can achieve a significant saving in bitrate
Useful for high bitrates

21. How Much Should We Compress Audio?
When applying audio compression, we must make a crucial decision as to how much we need it to be compressed.
This will affect the audio file size itself (like an mp3 music file) or the overall size of a video file (like Divx combined with mp3 audio).
Certain things need to be considered when deciding on this matter.
Generally, the idea is to preserve as much quality as possible, no matter what the purpose of the audio is.

22. Things To Consider
Let us first talk about compressing just an audio file, like mp3.
Assuming we ‘rip’ a WAV file from a CD and then compress it to mp3, the amount of compression is pretty much decided at 128 kb/s, 44 KHz, Stereo by default.
However, this has recently changed with improved bandwidth, storage space, portable players and the need for higher fidelity.
It is not uncommon for regular mp3 listeners to compress their CD tracks to 160 kb/s or 192 kb/s.

23. Things To Consider
Some even go as high as 224 kb/s or prefer to use a variable bitrate which changes between 96 kb/s and 224 kb/s – using a lower datarate when there is less information to encode in the waveform and a higher datarate when there is more information.
One way to go about it would be to see how much space and memory you have to spare, or can afford.
Then, think about what the audio file is going to be used for? Does the music even sound good enough to warrant a high bitrate. Sometimes, a mere instrumental with little variance can be effectively compressed to just 64 kb/s, Mono.

24. Things To Consider
Also take note of the actual space savings. If reducing the compression ratio saves you some 500 KB on a 700 MB CD, maybe it will be better to use the higher quality audio file because the benefit it gives to the user (listening pleasure) is worth more than your meager 500 KB saving.
As for applying audio compression when it is part of a video stream, you must realize that the priority here goes to the video.
Most of the file size (maybe 85%-95%) should be allocated to the video stream so that the picture quality is higher – this is what matters most to those who view video.

25. Things To Consider
But audio can’t be neglected altogether. It must be compressed to as small as possible without actually jeopardizing its ability to convey what is in the video.
For example, an MPEG-1 VCD by default uses 44.1 KHz, Stereo audio. If you are converting this to an MPEG-4 (or Divx) video file, you can easily reduce the audio to KHz, Stereo mp3, with no perceptible difference – especially if what you are encoding is a movie with dialogue and many parts that are silent. Sometimes, the movie is so old, just using mono is enough.
This is not the case with music videos, though. There, keeping the audio at 44.1 KHz might be essential. In these, you may have to allocate about 15%-20% for the audio stream.