1. Audio Fingerprinting MUMT 611 Philippe Zaborowski March 2005

2. What is Audio Fingerprinting? Every digital recording has unique features just like a human fingerprint. The audio fingerprint of a digital recording must first be extracted. Once extracted, it can be compared with reference fingerprints stored in a central database to identify the music.

3. Advantages of Fingerprinting Reduced storage requirements as fingerprints relatively small Efficient comparison as perceptual irrelevancies have been removed Efficient searching as the fingerprint database is also relatively small

4. Hash Functions vs. Fingerprinting In Cryptography hash functions allows comparison of two large digital files by comparing their hash values Very efficient way to determine whether or not a particular digital file is present in a large database

5. Hash Functions vs. Fingerprinting Hash functions cannot be used in Audio Fingerprinting: Changing the audio format will change the digital waveform (CD to mp3 conversion) Changing a few bits would result in a completely different hash value

6. Hash Functions vs. Fingerprinting Fingerprinting does not establish mathematical equality, but perceptual equality Perceptually similar digital recordings will result in similar but not an identical fingerprint X and Y are similar: ||F(X)-F(Y)|| < T X and Y are not similar: ||F(X)-F(Y)|| > T

7. Fingerprinting Parameters Robustness Reliability Fingerprint size Granularity Search speed Scalability

8. Applications Broadcast monitoring Connected Audio Database maintenance Digital rights management Music library organization

9. Fingerprint Extraction Semantic features Include: genre, beats-per-minute, mood Don't always have a clear meaning More difficult to compute Not universal (ex: BPM in classical music) Non-semantic features Spectral flatness measure (Fraunhofer) Loudness, bandwidth (Bonn et al) Energy at each band (Haitsma/Kalker)

10. Fingerprint Extraction (Haitsma) Overlapping Hanning windows are used to extract 32-bit sub-fingerprints every 11.6 ms Large overlap is used to smooth short varying differences in the waveforms FFT is used to find the band energy A fingerprint block consists of 256 sub- fingerprints about 3 seconds in length

11. Fingerprint Extraction (Haitsma)

13. Database Search (Haitsma) Brute force method for matching: 10,000 song database = 250 million sub- fingerprints Searching through every fingerprint would take 20 minutes! More efficient methods: Look up table is 800,000 times faster! Requires 2^32 extra memory for the LUT Average of 300 comparisons needed

14. Database Search (Haitsma)

15. Commercial Products Phillips (Haitsma) Audible Magic Relatable (Music Brainz) Gracenote

16. Conclusion Many practical applications for both industry and consumers Very accurate and reliable, but not yet proven in real world May force file sharing underground