1. Onset Detection in Audio Music J.-S Roger Jang ( 張智星 ) http://mirlab.org/jang MIR LabMIR Lab, CSIE Dept. National Taiwan University

2. -2- What Are Note Onsets?  Energy profile of a percussive instrument is modeled as ADSR stages  Note onset is the time where the slope is the highest, during the attack time.  Soft onsets via violin, etc, are much harder to define and detect.

3. -3- Difficulty in Onset Detection  Music types  Monophonic  Easier  Polyphonic  Harder  Instrument types  Percussive instruments  Easier  String instruments  Harder (soft onsets)

4. -4- Why Onset Detection is Useful?  It is a basic step in music analysis  Music transcription (from wave to midi)  Music editing (Song segmentation)  Tempo estimation  Beat tracking  Musical fingerprinting (the onset trace can serve as a robust id for fingerprinting)

5. -5- Onset Detection Function  ODF (onset detection function) creates a curve of onset strength, aka  Onset strength curve  Novelty curve  Most ODFs are based on time-frequency representation (spectrogram) of  Magnitude of STFT (Short-time Fourier transform)  Phase of STFT  Mel-band of STFT  Constant-Q transform

6. -6- ODF: Spectral Flux  Concept  sum the positive change in each frequency bin

7. -7- Flowchart of OSC  Steps of OSC  Spectrogram  Mel-band spectrogram  Spectral flux  Smoothed OSC via Gaussian smoothing  Trend of OSC via Gaussian smoothing  Trend-subtracted OSC  Check out wave2osc.m to see these steps.

8. -8- Example of OSC  Try “wave2osc.m”

9. -9- What Can You Do With OSC...  OSC  onsets  Pick peaks to have onsets  OSC  tempo (BPM, beats per minute)  Apply ACF (or other PDF) to find the BPM  OSC  beat tracking  Pick equal-spaced peaks to have beat positions

10. -10- Beat Tracking  Demos  http://mirlab.org/demo/beatTracking http://mirlab.org/demo/beatTracking  Try “beatTracking.m” in SAP toolbox

11. -11- Example of Beat Tracking  beatTracking.m

12. -12- Performance Indices of Beat Tracking  Many performance indices of BT  Check out audio beat tracking task of MIREX  Mostly adopted ones  Precision, recall, f- measure, accuracy  Try simSequence.m in SAP toolbox Precision = tp/(tp+fp)=3/(3+3) = 0.5 Recall = tp/(tp+fn)=3/(3+2) = 0.6 F-measure = tp/(tp+(fn+fp)/2)=3/(3+(2+3)/2) = 0.545 Accuracy = tp/(tp+fn+fp)=3/(3+2+3) = 0.375