2 IntroductionInstruments can be made in a variety of ways: think guitar, piano, organs, etc.Use electronic devices to create sounds: synthesisers.Can eitherRecreate an existing timbre… or something different.
3 IntroductionProducing a sound by sending an electrical signal to a speaker is trivial.The question is what are the relevant and desirable properties of the signal to ensure that the resulting sound is as desired (eg, similar to a real instrument).
4 Applications Musical instruments Computer games Sound effects for filmsMultimedia, computer system soundsMobile phonesSpeech synthesisToysEffects
5 Sound Synthesis Plan Synthesis I: Fundamentals Synthesis II: Additive Synthesis III: Filtering and distortionSynthesis IV: Other approachesPost-processing, pitch correction (autotunes)Sound perception
6 Sound Synthesis Part I: Fundamentals Nicolas Pugeault
7 Lecture Plan Introduction to sound synthesis Perception of sound LoudnessPitchTimbreSound synthesis – FundamentalsSummary
8 Sound (cont’d)Sound is a pattern of compression and depression of the airRecord it using microphonesPerceive it from our earsGenerate it by speaking or using speakersEnergy per m2 decreases with the square of distance...
9 Sound is a waveform Sound is a waveform, Can be reflected when hitting a non-transmissive surfaceIf the surface is flat, reflected in cohesive wayOtherwise depends on frequency and surface textureSound proof studio wall, forabsorbing high frequencies
11 The simplest sound: Pure tone Sinusoidal wave (440Hz)Periode p=1/f0
12 Reminder: Fourier Transform Idea: “All functions can be decomposed in a (possibly infinite) sum of sinusoidal functions of varying frequencies.”Transforms a function from time domain to frequency domain.Eg, right, for a square wave.FirstcomponentFirst twocomponentsFirst threecomponentsFirst fourcomponents
13 Loudness Often measured in decibels (dB) R=20*log10(A/A0) A0 is a reference amplitude, often taken as the threshold of audibility.Logarithmic perception of loudness. A change in 6dB means a doubling of amplitude!Range of audibility: ~120dB (1 to 1million)
14 Perception of Loudness Correlated with amplitude.Here:constant frequency (f0=440Hz)Varying amplitude (A = 0.2, 0.5 or 1.0)
15 Loudness (cont’d) However Perception of Loudness is frequency dependent.Sound X and Y have the same amplitude, which is louder, X or Y?X (100 Hz, A=1)Y (3,500Hz, A=1)Considering only amplitudes, sound Y should be the same loudness as sound X.However, Y is louder than X. Why?
16 Loudness (cont’d) Fletcher Munson (1933) Robinson & Dadson (1956) Subjects listen to pure tonesVarious frequenciesamplitude inc. per 10dBRobinson & Dadson (1956)more accurateBasis for standard ISO-226Perceived Loudness (Phons)1 Phon = 1dB 1kHzBritish Standard BS ISO 226 (2003) (source wikipedia)
17 Loudness (cont’d)There is a difference between sensory loudness and perceptual loudness! (Emmet, 1992)For the design of a synthesiser with large dynamical range, changing only amplitude is a poor choice since signal may clip.Solution: use spectral variation:Broad spectrum will likely result in a loud sound.Narrow spectrum will be perceived as quiet.
18 Perception of Pitch Frequency correlated with pitch Here: 3 examples of pure tones.What if sounds are more complex?Range: 20Hz-20kHzBest acuity: 200Hz-2kHz
19 Pitch: Fundamental & Harmonics Real sounds are not pure – more complex!The ear assumes that multiple frequency components form one sound.Harmonically related -> fuse into single pitch at Fundamental Frequency (f0 largest common divisor)Each sinusoid is called a Harmonic partial of the sound (fk = N*f0)
20 Fundamental & Harmonics (2) Fundamental f0First harmonic f1 = 2*f0Second harmonic f2 = 3*f0Third harmonic f3 = 4*f0...Seventh harmonic f7 = 7*f0
21 Fundamental & Harmonics (3) The pitch is correlated with the Fundamental frequency.Although in this example the fundamental is missing, the pitch is the same. The timbre is different.
22 Timbre All those sounds have the same pitch (A4, 440Hz) Flute A4Tuning fork A4Violin A4Singer A4They differ in timbre.
23 Defining TimbreDefinition (American Standard Association): “That attribute of sensation in terms of which a listener can judge that two sounds having the same loudness and pitch are dissimilar.” (ASA, 1960 ; Wikipedia, 2011)Has a “wastebasket” quality (Dixon Ward, 1965):What is neither loudness nor pitch...Synonyms: Tone quality or colour, texture...Affected by a sound’s envelope.
24 Timbre (cont’d) What physical parameters relate to timbre? Static spectrum (transient)Envelope of spectrum (transient)Dynamic spectrum (time-evolving)PhaseThis list is not exhaustive.cf “wastebasket” quality!
27 Timbre: Envelope (cont’d) FluteDifference in envelope (same note, 440Hz fundamental)Top: FluteBottom: Violin Envelope differs!Conclusion: Envelope is instrument-specific.Violin
28 Timbre: Envelope (cont’d) Arrows indicate formants.This slide indicates two speech vowels (i and u)Formants not only determine timbre but helps distinguishing vowels.(used in speech recognition)
29 Timbre: Dynamic Spectrum Will those two sounds have the same timbre?No, same average spectrum, but different timbre!Difference:Top: original soundBottom: time reversed.Conclusion: Temporal variation of spectrum impacts timbre!B
32 Timbre: Dynamic Spectrum (cont’d) A) NormalThis slides shows the long term (average) spectrum for two sounds (top: original and bottom: time reversed)Spectrum is identical; timbre is totally different very misleading!Conclusion: it is important to know how the spectrum evolves in time.The timbre does not only depends on the harmonic structure but on the way spectrum varies in time.B) Time reversed
33 Time envelope (ADSR) Time Envelope (ADSR) Attack is the time from nil to peak.Decay is the time from peak to the sustain level.Sustain is the level during the main sequence of the sound’s duration, until key is released.Release is the time to decay from sustain level to zero.
35 Time Envelope (example) Example of the same sound with and without attackAttack cut at 0.7s.With (blue+green):Without (green):
36 Timbre: Phase?Sound ASound BAre A and B of different timbres?
37 Timbre: Phase?Timbre depends (weakly) on phase relationship between harmonics.BUT waveforms are totally different, magnitude spectra identical, and timbre are (almost) identical!Conclusion: Human hearing is not sensitive to phase differences.Sound A: Square wave, fundamental 500Hz, 9 harmonics.Sound B: Square wave, fundamental 500Hz, 9 harmonics,every second harmonic phase shifted by 90 degrees.
38 Summary 1: Loudness Control In order to control loudness in synthetic sounds:Modify the spectral content:more energy at high frequency louder (see right).Modify the amplitudeHigher amplitude louder
39 Summary 2: Pitch Control Fundamental frequencyIn order to control pitch in synthetics sounds:Modify the fundamental frequency.High fundamental frequency high pitch.
40 Summary 3: Timbre Control In order to control timbre in synthetic sounds, modifySpectral contentSpectral envelopeSpectrum in timeSpectrum evolution during transient states
41 Plan Introduction to sound synthesis Perception of sound LoudnessPitchTimbreSound synthesis – FundamentalsSummary
42 Fundamental Definitions Computer Instrument: An algorithm that realizes (performs) a musical event.Unit Generator: A high-level “building block” in an instrument.
43 Oscillator Basic waveform generator Abbreviations: EG – Envelope GeneratorLFO – Low Frequency OscillatorVCA – Voltage Control AmplifierVCF – Voltage Control Filter
44 Abbreviations EG – Envelope Generator LFO – Low Frequency Oscillator VCA – Voltage Control AmplifierVCF – Voltage Control Filter
45 Two types of synthesisers Important TermsTwo types of synthesisersmonophonicpolyphonicYou can only play one note at a time. If you play several keys together, only one note will be generated no chords!You can play several notes at the same time can play chords!