1. Spectrum Analysis and Processing

2. What is SOUND?
Blend of elementary acoustic vibrations; combination of sine waves to create more complex sounds

3. Spectrum Analysis
Viewing the balance among various components of sound
Display of frequency content of a sound
Each component corresponds to air pressure variation rate
Spectrum analysis is useful in determining the spectral content of a sound. Spectrum analysis is not a process, but a means of determining sonic content

4. What is “Spectrum”?
Measures the distribution of signal energy as a function of frequency

5. Spectrum Plots
Reveal microstructure of vocal, instrumental and synthetic sounds
Reveal characteristic frequency energy of tones
Helps identify timbres, or at least characteristics of timbres
Often valuable in pitch and rhythm recognition

6. Spectrum Analysis (contd.)
Analyzing spectrum can also help modify/process sounds!
Data can be viewed and simply analyzed, OR it can be modified and resynthesized
EX:
Time Compression/expansion
Frequency shifting
Convolution Filtering and reverb effects
Cross Synthesis

7. Static Spectrum Plots Like a snapshot
2-dimensional image of amplitude vs. frequency
Measures average energy in each frequency region over the time period of the analyzed segment
Ex: PAZ Analyzer

8. Time-varying plot Depicts varying blend of frequencies over time
Plotted as 3 dimensional graph of spectrum vs. time
2 types
Waterfall (time axis moving in real time)
Sonogram or spectogram
Shows frequency vs. time
Frequency = vertical, time = horizontal; amplitude = darkness

9. Spectrum vs. Timbre Related concepts but not equivalent
Spectrum = physical property; distribution of energy as a function of frequency
Timbre = perceptual mechanism that classifies sound into families
Amplitude envelope (esp. attack)
Vibrato and tremolo undulations
Perceived loudness
Duration
Frequency content over time

10. Spectrum Analysis: History
Sir Isaac Newton coined term “spectrum” in 1781
describes bands of color
frequencies passing through a prism

11. Spectrum Analysis: History
Jean-Baptiste Joseph, Baron de Fourier
Fourier Theory
Complex vibrations can be analyzed as a sum of many simultaneous simple signals
Fourier Analysis = integer relationship between sinusoidal frequencies

12. Fourier Theorem
Fourier Theorem maintains that a function (or signal) can be described as the sum of a set of simple oscillating functions. Essentially says that a complex signal is made up of a sum of simple signals (see below).

13. Fourier Transform Mathematical procedure
Maps continuous (analog) waveform to a corresponding infinite Fourier series of elementary sinusoidal waves
Each wave has its on specific amplitude and phase
FT converts input signals into a spectrum representation!

14. Short Time Fourier Transform
Adaptation to sampled finite-duration time-varying signals
Same process applied to discrete (digital) signal

15. How does it work? Imposes sequence of TIME WINDOWS on input signal
Breaks signal into “short time” segments
Each based on a window function – non-negative and smooth bell- shaped curves
Window Function = specific envelope applied to each time window
Window duration = 1ms-1sec
Each window analyzed separately

16. About windows Used in many different types of processing
Granular synthesis – grain envelope through windowing function
GRM tools – various GRM tools operate using windowing functions to cut input signal into smaller segments
Windowing function essentially applies envelope to each segment of the divided signal

17. Window Types
All are quasi-bell shaped, and work well for general musical analysis/resynthesis
Hamming
Hanning
Gaussian
Kaiser
Blackmann-Harris
Example of Hamming

18. More Window Types!

19. Operation of STFT 1. Signal divided into WINDOWS
2. Discrete Fourier Transform (DFT) applied to each windowed segment
Transform algorithm applied to discrete (digital) signal
3. Output = discrete frequency spectrum
Measurement of energy at a set of specific, equally spaced frequencies

20. DFT Results Data generated by DFT = FRAME Contains MAGNITUDE SPECTRUM
Like frames in a film, sound is made up of individual frames of content
Contains MAGNITUDE SPECTRUM
Amplitude of each analyzed frequency component
Contains PHASE SPECTRUM
Initial phase value for each frequency component

21. STFT Procedure

22. STFT signals
Input
Windowed
Magnitude

23. Resynthesis from Analysis Data
Apply INVERSE DISCRETE FOURIER TRANSFORM (IDFT) to each frame
Windows are overlapped and added together to get resultant signal
That signal can then be exported as a new audio file

24. Phase Vocoder Popular sound analysis tool
Windowed input signal passes through bank of parallel band pass filters spread over frequency range
Similar to standard vocoding procedure.
Can be used for time stretching and pitch shifting
Phase vocoding decouples pitch and time so that each domain can be manipulated individually. In other words it allows pitch shifting without time stretching and time stretching without pitch shifting

25. Phase Vocoder Parameters
Frame Size
# of samples analyzed at one time
Larger frame size = greater frequency bins, lower time resolution
Smaller frame size = greater time resolution, less frequency bins
Strongly influences time calculation!
The affect this has on sound will become clearer when we look at Spear and Soundhack

26. Phase Vocoder Parameters
Window Type – window function shape
Select window shape from among standard types
Hamming, Hanning, Kaiser, truncated Gaussian, etc.
All quasi-bell shaped
Again, the purpose of the window type is to provide an envelope to each individual window

27. Phase Vocoder Parameters
FFT Size
FFT = fast Fourier transform
# of samples fed into algorithm
Usually nearest power of two that’s double the frame size
(so if frame size is 512, FFT size is 1024)

28. Phase Vocoder Parameters
Hop Size or Overlap Factor
Bin size = division of frequency into bands (or in this case bins); division by 50 Hz would result in 0-50Hz, Hz, Hz, etc.
Time advance from one frame to the next
Usually a fraction of the frame size
Overlap needed (usually 8x) to ensure accurate resynthesis
Greater overlap = greater accuracy = greater computation time

29. Hop size

30. Hop size, again

31. Overview
Frame size = number of input samples to be analyzed at one time; measured in samples per second
Window type = shape of window (Hanning, Hamming, etc.)
FFT size = total number of samples fed into the FFT algorithm
Hope size = amount of overlap of frames; amount of time taken between frames; more overlap equals more accurate timing!

32. Overview
Fourier transform – breaking a continuous signal into its corresponding spectral components
Short-time Fourier transform – breaking the input signal into “time windows” and then breaking each window into spectral components; allows for representation of time-varying signals
Discrete Fourier transform – simply a type of Fourier transform applied to discrete sampled signals (digital signals)
Fast Fourier transform – a fast and efficient method of employing the DFT process