2. Multimedia Authoring1 Introduction to Sound History A Brief History of Computer MusicA Brief History of Computer Music MIDI Analogue sound Digital audio

3. Multimedia Authoring2 MIDI Audio MIDI (Musical Instrument Digital Interface) Creating and playing MIDI music requires Sequencer software - to record and edit MIDI data Sound synthesizer (e.g. sound card) – to generate music MIDI keyboard (not necessary for playback only) MIDI is device dependent

4. Multimedia Authoring3 MIDI Audio General MIDI Included with each note to be played in a MIDI file is a patch number. This number defines which instrument is to play the sound. A GM – compatible patch map is most commonly used. 128 instruments and sound effects are included. http://people.virginia.edu/~pdr4h/gmpatch.html

5. Multimedia Authoring4 MIDI Audio Strengths and Weaknesses of MIDI Files Compact size Why are MIDI files so compact? Sequence of commands containing instructions about how to generate music (some similarities with sheet music) Simple to change the instrument playing the sound Cannot be used to generate arbitrary sounds Instrumental music only Use MIDI if Digital audio not suitable due to lack of memory/bandwidth Have high quality sound source Have control over playback Don’t need spoken dialog

6. Multimedia Authoring5 Analogue Sound What is a Sound Wave? Animation courtesy of Dr. Dan Russell, Kettering University http://paws.kettering.edu/~drussell/Demos/waves-intro/waves- intro.html

7. Multimedia Authoring6 Analogue Sound

8. Multimedia Authoring7 Analogue Sound

9. Multimedia Authoring8 Analogue Sound Characteristics of Sound Waves Wavelength; this is the distance from the crest of one wave to the crest of the next. Frequency; this is the number of waves that pass a point in each second. Amplitude; this is the measure of the amount of energy in a sound wave.

10. Multimedia Authoring9 Analogue Sound Rapid vibrations of air molecules create a high-pitched sound (treble); a slower rate of vibration creates a low-pitched sound (bass) (Beggs, Josh and Thede, Dylan, 2001). Frequency

11. The frequency of a sound tells us how fast the air is moving, and is measured in Hertz (Hz). Most people can hear frequencies in the range 100 Hz-15 000 Hz. 60 Hz | 100 Hz | 440Hz | 500 Hz | 1 000 Hz | 8 000 Hz | 10 000 Hz | 12 000 Hz | 15 000 Hz Human ears are most sensitive in a band from 2 000 Hz to 5 000 Hz, and being able to hear in this range is important to being able to understand speech. 2 000 Hz | 4 000 Hz | 5 000 Hz A few people can hear very high frequencies above 19 000 Hz. 20 000 Hz Multimedia Authoring10 Analogue Sound

12. Multimedia Authoring11 Analogue Sound Amplitude Schmidt-Jones, C. Amplitude and Dynamics, Connexions Web site. http://cnx.org/content/m12372/1.2/, Jan 5, 2006. The size of a wave (how much it is "piled up" at the high points) is its amplitude. For sound waves, the bigger the amplitude, the louder the sound.

13. Multimedia Authoring12 Analogue Sound Level The level of sound, known as volume, is measured in decibels (dB). 4 000 Hz : maximum level 0.0 dB 4 000 Hz : half power -6.0 dB 4 000 Hz : very quiet -18.0 dB 4 000 Hz : getting too loud (starting to distort) +6.0 dB Use the VU (volume) meter in GarageBand so that the loudest part of your pieces approach, or just get into, the red range.

14. Multimedia Authoring13 Analogue Sound Analogue Recording and Playback A direct representation of sound waves are stored. For example, Thomas Edison's phonograph recorded sound on tinfoil wrapped around a cardboard cylinder. A horn focused the sound waves onto a thin membrane attached to a needle. As the cylinder rotated, the needle cut a continuous groove into the tinfoil. The air pressure variations of the sound waves caused the diaphragm and needle to move up and down, varying the depth of the groove to create a recording. Vinyl records use basically the same principle, except the needle moves from side to side. To playback, the needle was placed at the start of the groove and the cylinder rotated. As the needle traced the path of the groove around the cylinder, the varying depth of the groove moved it up and down. The needle caused the diaphragm to move the same amount as it did when the recording was made, reproducing the original sound waves.

15. Multimedia Authoring14 Analogue Sound Analogue Audio Signals Analogue audio signals are the electronic version of a groove on a record. Variations in voltages on a wire are like the variations in the depth (or width) of a groove. They both represent the air pressure variations of sound. Microphones convert air pressure variations of sound waves into a varying voltage sound signal. Magnetic tape records sound by representing variations in sound waves as variations in the amount of magnetism stored in a metallic coating on a thin strip of plastic tape. Speakers have a coil of wire into which the sound signal is fed. The coil is surrounded by a magnet. Variations in the signal cause the coil to move and a diaphragm attached to the coil creates air pressure variations to form sound waves.

16. Multimedia Authoring15 Digital Sound Digitising Sound We can digitise sound from any source - natural or prerecorded. In digital audio, sound is represented by a sequence of numbers that correspond to the signal level at a predetermined interval. The signal is made up of bits, where 0 represents low (off) and 1 high (on - close to maximum voltage). To hear the original sound, digital signals are converted back to analog as speakers recreate sound waves using a digital to analog (D/A) converter e.g. there is a D/A in CD players and sound cards.

17. Multimedia Authoring16 Digital Sound If we digitise sound, we are sampling sound. The process converts an analog signal to a digital by using an analog to digital (A/D) converter. The quality of digital recordings depend on the sampling rate and the bitdepth. Every nth of a second, a sample of sound is taken and stored as digital information in bits and bytes. Sampling Rate: frequency measured in kilohertz Bitdepth: how many numbers are used to represent the value of each sample - also known as sample size, resolution, or dynamic range. The more often you take a sample and the more data you store about that sample, the finer the resolution and quality when played back. Sound Samples

18. Multimedia Authoring17 Digital Sound How Sampling Works Figure 5-2, Vaughan (2008) NB. The original waveform cannot be reconstructed if the sampling frequency is too low!

19. Multimedia Authoring18 Digital Sound Sound Sampling Examples 4000 Hz 4 000 Hz sample rate = 8 000 Hz 4 000 Hz sample rate = 11 025 Hz 4 000 Hz sample rate = 16 000 Hz 4 000 Hz sample rate = 22 050 Hz 4 000 Hz sample rate = 32 000 Hz 4 000 Hz sample rate = 44 100 Hz

20. Multimedia Authoring19 Digital Sound Professional recording studios use 96 000 Hz sampling frequency and mix it down to CD quality. What is the sampling rate that gives CD quality? To record a certain frequency we need at least (but preferably more than) two samples for each period to accurately record the peaks and troughs. So we need a rate at least twice as high as the highest frequency to be recorded. What is the highest frequency people are likely to hear? What sample rate is just greater than twice that frequency? If we use a sample rate of 96 000 Hz, will we be able to hear 48 000 Hz? Using this sample rate, we will be able to record more than four samples for each 20 000 Hz period. Therefore, the chance of losing high frequencies dramatically reduces. The three sampling frequencies most often used in multimedia are: 1.11 025 Hz 2. Hz 3. Hz Some equipment and software allows up to 192 000 Hz and this will probably soon become the professional standard. The advantage of the higher sampling rates are much better sound quality. Suggest a disadvantage. Sound Samples

21. Multimedia Authoring20 Digital Sound The resolution of a digital signal is the number of distinct integer values available to represent the voltage level of an analog signal. The exact voltage of a sample is rounded off to the nearest integer. The more integers the higher the resolution and so the more accurately the voltage can be represented. CD audio uses 16 bits per sample (16-bit resolution). How many possible integer values? Professional recording studios use systems with 24-bit resolution. High sampling rates have little effect if the resolution is too low. Resolution

22. Multimedia Authoring21 Digital Sound

23. Multimedia Authoring22 Digital Sound Because computers process integers much more efficiently than floating point (decimal) numbers, the value of each sample is rounded to the nearest integer. Because the voltage of an analog signal varies continuously, the values measured for most samples will not be whole numbers. So the A/D converter rounds the value of each sample to the nearest whole number. The range of possible values is determined by the resolution of the signal. Quantization

24. Multimedia Authoring23 Digital Sound Clipping: If the amplitude is greater than the intervals available, clipping of the top and bottom of the wave occurs. This can greatly distort it. The maximum level is 0 dB. Other levels are negative. If the average level of a signal goes above zero it is clipped. It is usual to set average signal levels a bit below maximum to allow for unexpected peaks. Unwanted background hissing: can be produced by quantization. Quantization noise - noise generated by (bad) quantization. The first digital pianos did not sell well - not much memory so not very dynamic. Also, there was a rough edge at the end of samples. The conversion of a piano sample (that slowly fades away) with 65536 levels to 8 bits (how many levels) gave a very poor result. Remedy - dithering While sampling the piano, the soundcard adds a little noise to the signal (3-6 dB). This helps the signal become a little louder. We do not hear the noise, which is very soft and does not change as much as the recorded signal - ears overlook it. Problems with Quantization

25. Multimedia Authoring24 Digital Sound Quantization Figure 4-2, Vaughan (2011)

26. Multimedia Authoring25 Digital Sound The bit-rate is the number of bits per second used to represent a signal. It has a direct correlation to file size and quality. e.g. Bit rate of uncompressed audio: bit rate = sampling rate * resolution * channels e.g. 44,100 * 16 * 2 = 1,411,200 bps Since the quality of your audio is based on the quality of your recording and not the quality of the device that your end user will play the audio, digital audio is said to be device independent. Bit Rate

27. Multimedia Authoring26 Digital Sound The bit-rate is the number of bits per second used to represent a signal. It has a direct correlation to file size and quality. e.g. Bit rate of uncompressed audio: bit rate = sampling rate * resolution * channels e.g. 44,100 * 16 * 2 = 1,411,200 bps Since the quality of your audio is based on the quality of your recording and not the quality of the device that your end user will play the audio, digital audio is said to be device independent. Bit Rate

28. Multimedia Authoring27 Digital Sound Space required for stereo digital audio (uncompressed) Bit Depth Sample Rate (Hz) Bit Rate (Mbit/sec) File Size of one stereo minute (MB) File size of a three minute song (MB) 1644,1001,3510.130.3 1648,0001.4611.033 2496,0004.3933.099

29. Multimedia Authoring28 Digital Sound 1.When we press the record button, the soundcard starts a very accurate stopwatch (sample rate). 2.low pass, anti-aliasing filter: very high frequencies that sound card cannot handle cut are off (maximum frequency that can be recorded at a certain sample rate is half that rate - called the Nyquist frequency) reduces the sound quality without it, the sound would be more seriously damaged (become unrecognizable) low pass - lets the low frequencies pass through anti-aliasing - smoothes, blurs filter - takes away some part and leaves the rest 3.Every time stopwatch finishes a cycle, the analog to digital converter (ADC) in the soundcard looks at the filtered input signal and calculates how loud it as at that exact moment and transforms the loudness level to the nearest digital number, which is stored in memory or secondary memory. Digitizing Process

30. Multimedia Authoring29 Multimedia Authoring References A brief history of computer music http://adagio.calarts.edu/~eric/cm.html Sound Waves http://www.fi.edu/fellows/fellow2/apr99/soundvib.html Digital Audio http://emeld.org/workshop/2003/Bartek-demo.html How analogue and digital recording works http://electronics.howstuffworks.com/analog-digital1.htm Vaughan, T. (2011) Multimedia: Making It Work, 8 th Ed., New York: McGraw-Hill