1. Jonah Shifrin, Bryan Pardo, Colin Meek, William Birmingham
Hidden Markov Models
HMM-Based Musical Query Retrieval
Jonah Shifrin, Bryan Pardo, Colin Meek, William Birmingham
Student: Leela Krishna Kadiri

2. Introduction
A model that explicitly maintains a probability distribution over the set of possible observations for each state is called a hidden Markov model (HMM). More formally, an HMM requires two things in addition to that required for a standard Markov model:
A set of possible observations, O={o1, o2, o3,…, on}.
A probability distribution over the set of observations for each state in S.
The paper describes a way in which music could be retrieved by providing a musical query as a piece.
Method used – HMM(Hidden Markov Model)

3. The Method
Pieces in the database are represented as hidden Markov models (HMMs). The query is treated as an observation sequence and a model is ranked similar to the query if its HMM has a high likelihood of generating the query.
The query is sung by the user and is recorded in .wav format.
The query is transcribed into MIDI format.
A sequence of values are derived from the MIDI representation (the deltaPitch, IOI and IOIratio values)

4. The Method A note transition between note n and note n+1 is described
by the duple <deltaPitch, IOIratio>.
deltaPitchn is the difference in pitch between note n and note n+1.
The inter onset interval (IOIn) is the difference between the onset of notes n and n+1.
IOIration is IOIn/IOIn For the final transition, IOIn = IOIn/durationn+1.

5. The Method

6. The Method
The songs in the database are represented as themes. These themes are represented as HMMs.
For the HMM construction, the states are represented by note transitions, eg, 8 note transitions can be represented by 4 unique duples of <deltaPitch,IOIratio> .
Here, the nodes are represent the states, the transitions by arrows, the value below each node indicates that the traversal may begin at that state and the numerical value on the edges are transition probabilities.

7. States
Each HMM is built automatically from a MIDI file encoding the theme. The unique duples characterizing the note transitions found in the MIDI file form the states in the model.
In the previous example shown a passage with eight note transitions characterized by four unique duples. Each unique duple is represented as a state.

8. Hidden Markov Model
The model is a weighted automaton that consists of:
A set of states, S = {s1, s2, s3,…, sn}.
A set of transition probabilities, T, where each ti,j in T represents the probability of a transition from si to sj.
A probability distribution, π, where πi is the probability the automaton will begin in state si.
E, a subset of S containing the legal ending states.
A set of possible observations, O={o1, o2, o3,…, on}.
A probability distribution over the set of observations for each state in S.

9. Hidden Markov Model
In this method of music retrieval all states are considered as legal ending states.
The probability distribution for the initial state of each model in the database is given by the formula, where |S| is the number of states, p is probability that the sung query exactly represents the theme in the database.

10. Estimating Observation Probabilities
In a general voice sung query there are 25 deltaPitch values and 27 IOIratio values given the hidden states. But, the hidden states depend on these values of the duple <deltaPitch, IOIratio>.
Two different observation-hidden state pair tables are considered of values 25*25 and 27*27 and given the conditional independence, the probability of encountering an observational duple given a hidden state is given by,

11. Finding The target
Given the observation sequence and the set of models(themes), the Forward Algorithm produces a value between 0 and 1 predicting whether the HMM produced the observation.
Scaling the value with a constant and displaying the results in a descending order, we can obtain the correct song that was sung by the user.

12. Results
The results are compared to Baseline matcher and the proposed system outperformed the baseline matcher in 8 out of ten cases.

13. HMM
HMMs are employed in this problem to mainly reduce the pitch errors that are introduced by the singers.
The variations of HMM like the pairHMM is a more complex yet efficient way of producing the possible states with a difference that it produces two sequences of states simultaneously.

14. References
Jonah Shifrin, Bryan Pardo, Colin Meek, William Birmingham HMM Based Musical Query Retrieval JCDL 02, July 13-17, 2002, Portland, Oregon, USA, Copyright 2000 ACM /02/0007