Spectrum Analysis and Processing

What is SOUND? Blend of elementary acoustic vibrations; combination of sine waves to create more complex sounds

Spectrum Analysis Viewing the balance among various components of sound Display of frequency content of a sound Each component corresponds to air pressure variation rate Spectrum analysis is useful in determining the spectral content of a sound. Spectrum analysis is not a process, but a means of determining sonic content

What is “Spectrum”? Measures the distribution of signal energy as a function of frequency

Spectrum Plots Reveal microstructure of vocal, instrumental and synthetic sounds Reveal characteristic frequency energy of tones Helps identify timbres, or at least characteristics of timbres Often valuable in pitch and rhythm recognition

Spectrum Analysis (contd.) Analyzing spectrum can also help modify/process sounds! Data can be viewed and simply analyzed, OR it can be modified and resynthesized EX: Time Compression/expansion Frequency shifting Convolution Filtering and reverb effects Cross Synthesis

Static Spectrum Plots Like a snapshot 2-dimensional image of amplitude vs. frequency Measures average energy in each frequency region over the time period of the analyzed segment Ex: PAZ Analyzer

Time-varying plot Depicts varying blend of frequencies over time Plotted as 3 dimensional graph of spectrum vs. time 2 types Waterfall (time axis moving in real time) Sonogram or spectogram Shows frequency vs. time Frequency = vertical, time = horizontal; amplitude = darkness

Spectrum vs. Timbre Related concepts but not equivalent Spectrum = physical property; distribution of energy as a function of frequency Timbre = perceptual mechanism that classifies sound into families Amplitude envelope (esp. attack) Vibrato and tremolo undulations Perceived loudness Duration Frequency content over time

Spectrum Analysis: History Sir Isaac Newton coined term “spectrum” in 1781 describes bands of color frequencies passing through a prism

Spectrum Analysis: History Jean-Baptiste Joseph, Baron de Fourier 1822 - Fourier Theory Complex vibrations can be analyzed as a sum of many simultaneous simple signals Fourier Analysis = integer relationship between sinusoidal frequencies

Fourier Theorem Fourier Theorem maintains that a function (or signal) can be described as the sum of a set of simple oscillating functions. Essentially says that a complex signal is made up of a sum of simple signals (see below).

Fourier Transform Mathematical procedure Maps continuous (analog) waveform to a corresponding infinite Fourier series of elementary sinusoidal waves Each wave has its on specific amplitude and phase FT converts input signals into a spectrum representation!

Short Time Fourier Transform Adaptation to sampled finite-duration time-varying signals Same process applied to discrete (digital) signal

How does it work? Imposes sequence of TIME WINDOWS on input signal Breaks signal into “short time” segments Each based on a window function – non-negative and smooth bell- shaped curves Window Function = specific envelope applied to each time window Window duration = 1ms-1sec Each window analyzed separately

About windows Used in many different types of processing Granular synthesis – grain envelope through windowing function GRM tools – various GRM tools operate using windowing functions to cut input signal into smaller segments Windowing function essentially applies envelope to each segment of the divided signal

Window Types All are quasi-bell shaped, and work well for general musical analysis/resynthesis Hamming Hanning Gaussian Kaiser Blackmann-Harris Example of Hamming

More Window Types!

Operation of STFT 1. Signal divided into WINDOWS 2. Discrete Fourier Transform (DFT) applied to each windowed segment Transform algorithm applied to discrete (digital) signal 3. Output = discrete frequency spectrum Measurement of energy at a set of specific, equally spaced frequencies

DFT Results Data generated by DFT = FRAME Contains MAGNITUDE SPECTRUM Like frames in a film, sound is made up of individual frames of content Contains MAGNITUDE SPECTRUM Amplitude of each analyzed frequency component Contains PHASE SPECTRUM Initial phase value for each frequency component

STFT Procedure

STFT signals Input Windowed Magnitude

Resynthesis from Analysis Data Apply INVERSE DISCRETE FOURIER TRANSFORM (IDFT) to each frame Windows are overlapped and added together to get resultant signal That signal can then be exported as a new audio file

Phase Vocoder Popular sound analysis tool Windowed input signal passes through bank of parallel band pass filters spread over frequency range Similar to standard vocoding procedure. Can be used for time stretching and pitch shifting Phase vocoding decouples pitch and time so that each domain can be manipulated individually. In other words it allows pitch shifting without time stretching and time stretching without pitch shifting

Phase Vocoder Parameters Frame Size # of samples analyzed at one time Larger frame size = greater frequency bins, lower time resolution Smaller frame size = greater time resolution, less frequency bins Strongly influences time calculation! The affect this has on sound will become clearer when we look at Spear and Soundhack

Phase Vocoder Parameters Window Type – window function shape Select window shape from among standard types Hamming, Hanning, Kaiser, truncated Gaussian, etc. All quasi-bell shaped Again, the purpose of the window type is to provide an envelope to each individual window

Phase Vocoder Parameters FFT Size FFT = fast Fourier transform # of samples fed into algorithm Usually nearest power of two that’s double the frame size (so if frame size is 512, FFT size is 1024)

Phase Vocoder Parameters Hop Size or Overlap Factor Bin size = division of frequency into bands (or in this case bins); division by 50 Hz would result in 0-50Hz, 50-100Hz, 100-150Hz, etc. Time advance from one frame to the next Usually a fraction of the frame size Overlap needed (usually 8x) to ensure accurate resynthesis Greater overlap = greater accuracy = greater computation time

Hop size

Hop size, again

Overview Frame size = number of input samples to be analyzed at one time; measured in samples per second Window type = shape of window (Hanning, Hamming, etc.) FFT size = total number of samples fed into the FFT algorithm Hope size = amount of overlap of frames; amount of time taken between frames; more overlap equals more accurate timing!

Overview Fourier transform – breaking a continuous signal into its corresponding spectral components Short-time Fourier transform – breaking the input signal into “time windows” and then breaking each window into spectral components; allows for representation of time-varying signals Discrete Fourier transform – simply a type of Fourier transform applied to discrete sampled signals (digital signals) Fast Fourier transform – a fast and efficient method of employing the DFT process