Easily extensible unix software for spectral analysis, display modification, and synthesis of musical sounds James W. Beauchamp School of Music Dept.

Slides:



Advertisements
Similar presentations
Acoustic/Prosodic Features
Advertisements

Senior Capstone Design Project Real-Time Electrocardiogram Spectral Analyzer Ryan Carnathan, Keith Berry, Mark Stadick, & Greg Michaelson Electrical &
Sound Synthesis Part II: Oscillators, Additive Synthesis & Modulation.
Department of Kinesiology and Applied Physiology Spectrum Estimation W. Rose
Speech Processing AEGIS RET All-Hands Meeting University of Central Florida July 20, 2012 Applications of Images and Signals in High Schools.
Voiceprint System Development Design, implement, test unique voiceprint biometric system Research Day Presentation, May 3 rd 2013 Rahul Raj (Team Lead),
Thursday, October 12, Fourier Transform (and Inverse Fourier Transform) Last Class How to do Fourier Analysis (IDL, MATLAB) What is FFT?? What about.
Time-scale and pitch modification Algorithms review Alexey Lukin.
Page 0 of 34 MBE Vocoder. Page 1 of 34 Outline Introduction to vocoders MBE vocoder –MBE Parameters –Parameter estimation –Analysis and synthesis algorithm.
Loris for Your Cough Roshan Mansinghani, Esmeralda Martinez, James McDougall, Travis McPhail Results: The noise frequencies were completely removed including.
Analog Data Processing with BioProc3 Part Two EMG Analysis Techniques.
Spectrum Analyzer. Another Oscilloscope??? Like an oscilloscope Oscilloscope in time domain Spectrum analyzer in frequency domain (selectable)
PH 105 Dr. Cecilia Vogel Lecture 13. OUTLINE  Timbre and graphs:  Time graph  Spectrum graph  Spectrogram  Envelope  scales  units  interval factors.
System Microphone Keyboard Output. Cross Synthesis: Two Implementations.
Effects in frequency domain Stefania Serafin Music Informatics Fall 2004.
Additional Notes on Wavetable Synthesis R.C. Maher ECEN4002/5002 DSP Laboratory Spring 2002.
Music Processing Roger B. Dannenberg. Overview  Music Representation  MIDI and Synthesizers  Synthesis Techniques  Music Understanding.
Spectrum Analysis Sound AnalysisSound Analysis What are we going to do?What are we going to do? Record a soundRecord a sound recorded sound analog-to-digital.
Voice Transformations Challenges: Signal processing techniques have advanced faster than our understanding of the physics Examples: – Rate of articulation.
EE 198 B Senior Design Project. Spectrum Analyzer.
Warped Linear Prediction Concept: Warp the spectrum to emulate human perception; then perform linear prediction on the result Approaches to warp the spectrum:
Continuous-Time Signal Analysis: The Fourier Transform
Human Psychoacoustics shows ‘tuning’ for frequencies of speech If a tree falls in the forest and no one is there to hear it, will it make a sound?
Recent Research in Musical Timbre Perception James W. Beauchamp University of Illinois at Urbana-Champaign Andrew B. Horner Hong University of Science.
8/9/2015 Frequency Domain Methods. Time domain worldFrequency domain worldFourier Transform: F Inverse Fourier Transform: F --1 Oscilloscope Spectrum.
Representing Acoustic Information
Harmonics, Timbre & The Frequency Domain
infinity-project.org Engineering education for today’s classroom 53 Design Problem - Digital Band Build a digital system that can create music of any.
Goals For This Class Quickly review of the main results from last class Convolution and Cross-correlation Discrete Fourier Analysis: Important Considerations.
Vibrationdata 1 Unit 5 The Fourier Transform. Vibrationdata 2 Courtesy of Professor Alan M. Nathan, University of Illinois at Urbana-Champaign.
Digital Sound and Video Chapter 10, Exploring the Digital Domain.
GCT731 Fall 2014 Topics in Music Technology - Music Information Retrieval Overview of MIR Systems Audio and Music Representations (Part 1) 1.
Lecture 1 Signals in the Time and Frequency Domains
MIDI. A protocol that enables computers, synthesizers, keyboards, and other musical devices to communicate with each other. Instead of storing actual.
Instrument Recognition in Polyphonic Music Jana Eggink Supervisor: Guy J. Brown University of Sheffield
Acoustic Analysis of Speech Robert A. Prosek, Ph.D. CSD 301 Robert A. Prosek, Ph.D. CSD 301.
Chapter 16 Speech Synthesis Algorithms 16.1 Synthesis based on LPC 16.2 Synthesis based on formants 16.3 Synthesis based on homomorphic processing 16.4.
Speech Signal Processing I By Edmilson Morais And Prof. Greg. Dogil Second Lecture Stuttgart, October 25, 2001.
Music-Driven Motion Editing Marc Cardle Rainbow Group Computer Laboratory University of Cambridge Marc Cardle Rainbow Group Computer.
Pre-Class Music Paul Lansky Six Fantasies on a Poem by Thomas Campion.
ECE 5525 Osama Saraireh Fall 2005 Dr. Veton Kepuska
Judith C. Brown Journal of the Acoustical Society of America,1991 Jain-De,Lee.
(Extremely) Simplified Model of Speech Production
MSc Project Musical Instrument Identification System MIIS Xiang LI ee05m216 Supervisor: Mark Plumbley.
MDO4000C Series vs. Regular Scope FFTs Competitive Fact Sheet Benefits: ~20dB better dynamic range than a scope FFT RF support.
FUNCTIONS REVIEW PRE-CALCULUS UNIT 1 REVIEW. STANDARD 1: DESCRIBE SUBSETS OF REAL NUMBERS What are the categories? Where would these be placed? 7, 6,
The Discrete Fourier Transform
The Story of Wavelets Theory and Engineering Applications
Time Compression/Expansion Independent of Pitch. Listening Dies Irae from Requiem, by Michel Chion (1973)
Music Transcription through Statistical Analysis Group 3 Austin Assavavallop, William Feater, Greg Heim, Philipp Pfieffenberger, Wamba Yves Design Phase.
Signal acquisition A/D conversion Sampling rate  Nyquist-Shannon sampling theorem: If bandlimited signal x(f) holds in [-B;B], then if f s = 1 / T.
Project Presentation Eoin Culhane Multi Channel Music Recognition for an Electric Guitar.
Intro to Fourier Series BY JORDAN KEARNS (W&L ‘14) & JON ERICKSON (STILL HERE )
Time-Frequency Spectrum
CS 591 S1 – Computational Audio
Spectrum Analysis and Processing
CS 591 S1 – Computational Audio
PRISM: PROCESSING AND REVIEW INTERFACE FOR STRONG MOTION DATA SOFTWARE
Vocoders.
III Digital Audio III.9 (Wed Oct 25) Phase vocoder for tempo and pitch changes.
Unit 5 The Fourier Transform.
Sampling and Reconstruction
1 Vocoders. 2 The Channel Vocoder (analyzer) : The channel vocoder employs a bank of bandpass filters,  Each having a bandwidth between 100 HZ and 300.
Intro to Fourier Series
CSCE 643 Computer Vision: Thinking in Frequency
III Digital Audio III.9 (Wed Oct 24) Phase vocoder for tempo and pitch changes.
LECTURE 18: FAST FOURIER TRANSFORM
INTRODUCTION TO THE SHORT-TIME FOURIER TRANSFORM (STFT)
LECTURE 18: FAST FOURIER TRANSFORM
Review and Importance CS 111.
Presentation transcript:

Easily extensible unix software for spectral analysis, display modification, and synthesis of musical sounds James W. Beauchamp School of Music Dept. of Electrical and Computer Engineering University of Illinois at Urbana-Champaign

Talk Topics SNDAN features –time-domain utilities –phase vocoder harmonic spectrum analysis –spectral graphics, modification, resynthesis –frequency tracking analysis, graphics, synthesis –pitch detection, conversion to harmonic format SNDAN applications Future developments Conclusions

SNDAN OVERVIEW: ANALYSIS FRONT END signal GRAPHICSMODIFICATIONRESYNTHESIS analysis data

SNDAN OVERVIEW: SIGNAL VIEWING AND EDITING

SNDAN OVERVIEW: SPECTRUM ANALYSIS

SNDAN OVERVIEW: SPECTRUM VIEWING, MODIFICATION, AND RESYNTHESIS

Phase Vocoder Analysis s’(t) HAMMING WINDOW (double period) DISCRETE FOURIER TRANSFORM (FFT) w(t)s’(t) OVERLAP BY 1/2 PERIOD THROW AWAY K/2, K ODD COMPUTE AMPLITUDES AND PHASES COMPUTE FREQUENCY DEVIATIONS harmonic data BANDLIMITED INTERPOLATION RESAMPLER s(t)s(t) f s (sample frequency) f a (analysis frequency) sound signal

Harmonic Data Graphics EPS graphics display harmonic data 1D: Amplitude vs. Frequency (snapshot bar, line; comp. overlay) 1D & 2D: Frequency vs. Time (individual, spectrogram) 3D: Amplitude vs. Frequency and Time Inharmonicity vs. Time Spectral Centroid vs. Time Spectral Centroid vs. RMS Ampl. Spectral Irreg. vs. Time Inverse Spectral Density vs. Time Musical Pitch vs. Time

Example 2D graph

Example 3D graph

Harmonic Data Modification harmonic data Smooth A k vs. time (t) Make A k (t) proportional to A rms (t) Smooth A k vs. frequency (k) Scale A k by k p to achieve new average centroid Scale A k to achieve designated spectrum or aux. spectrum aux. harmonic data Warp attack time Reduce duration without affecting attack and decay. Smooth f k vs. time (t) Make all f k (t) harmonic to f ave (t) Flatten f k to average or harmonic Quantize fund. freq. to ET pitch

Harmonic Data Resynthesis harmonic data synthetic signal AMPLITUDE & FREQUENCY LINEAR INTERPOLATION (scale amplitude, freq, duration) AMPLITUDE & PHASE QUADRATIC INTERPOLATION

Signal Modification Example Original Flute Time-smoothed Amplitudes Time-smoothed Amplitudes & Frequencies Time-smoothed Amplitudes & Flattened Frequencies Time-smoothed, RMSed, & Spectrum Envelope Smoothed & Flattened Frequencies Time-smoothed, Spectrum Envelope Smoothed & Flattened Frequencies

Frequency Tracking (MQ) Data Analysis KAISER WINDOW WITH 100% ZERO FILL s(t)s(t) f s f min sound signal FFT WITH TYPICAL 6 MS HOP APPLY THRESHOLD IDENTIFY AND SAVE SPECTRAL PEAKS AT EACH FRAME COMPUTE EACH PEAK’S AMPLITUDE, FREQUENCY AND PHASE partial (MQ) data A thresh CONNECT PEAKS TO NEXT FRAME PEAKS (TRACKS)

Graphics partial data 2D: FREQUENCY VS. TIME 3D: AMPLITUDE VS. FREQUENCY VS. TIME EPS display Synthesis synthetic signal INTERPOLATION: AMPLITUDE - LINEAR PHASE - CUBIC partial data time scale Partial Data Processing

Pitch Detection TWO-WAY MISMATCH HARMONIC MATCHING METHOD fundamental frequency data F 0 (t) f min f max partial data HARMONIC SIEVE fund freq data F 0 (t) harmonic data n har (no. of harmonics) frequency tolerance Harmonic Separation

Saxophone Solo Pitch Plot:2D Peak Track Plot: Original SoundSynthesized from Partial Data Synthesized from Harmonic-reduced Data Partial Data Stretched x2 Harmonic Data Stretched x2 Harmonic Data Stretched x2 Smoothed Freq

Applications Synthesis instrument development –nonlinear and frequency modulation –wavetable trumpet and piano Timbre investigations –simplified sounds for discrimination studies –normalized sounds for MDS studies –perturbed sounds for discrimination studies –synthesis quality evaluation Music composition using Music 4C

Future Developments More features for partial data format Integrate programs into single program More advanced analysis front end Multi-Channel Create GUI interface Real time Port to more platforms

SNDAN Conclusions Provides analysis, graphics, modification, and synthesis Specialized for musical sounds Two spectrum data formats: harmonic and partial Contains pitch detector Unix source code modular and easily extensible Source code available at: – DOS binary version available at: – Real-time GUI spinoff analyzer for Mac available at: –