1. 1 Music Classification Using Significant Repeating Patterns Chang-Rong Lin, Ning-Han Liu, Yi-Hung Wu, Arbee L.P. Chen, Proc. of 9th International Conference, Database Systems for Advances Applications (DASFAA 2004) Adviser:Jia-Ling Koh Speaker: Yu-ting Kung

2. 2 Introduction Automatic classification of music data becomes a critical technique for providing an efficient and effective retrieval of music data In this paper, propose a new approach for classifying music data based on their contents: 1. Rhythm 2. melody

3. 3 Introduction (Cont.) Moreover, use Repeating Patterns of music data to do music classification. The repeating pattern that satisfy the constraints are called significant repeating patterns (SRP) This approach contains two stages 1. Feature extraction 2. SRP-based classification

4. 4 Introduction (Cont.) The flowchart of this approach Classified musicMusic to be classified Feature extraction Representation of music (melody & rhythm) Generation of Significant repeating patterns Usefulness of SRP for classification Similarity measure for SRP matching Importance of Source SRP SRP-based classification Relevance of source SRP Class determination

5. 5 Feature Extraction Representations of Music 1. Rhythm: sequence of beats in music According to the duration of a note, classify each note into one of the types in rhythm Ex: SymbolDurationSymbolDurationSymbolDuration A(0. 1/4]B(1/4. 2/4]C(2/4. 3/4] D(3/4. 4/4]E(4/4. 5/4]F(5/4. 6/4] G(6/4. 7/4]H(7/4. 8/4]I Above 2 beat

6. 6 Feature Extraction (Cont.) 2. Melody: sequence of pitches in music According to the length of a pitch interval, classify each pitch interval into one of the types in melody Ex: Symbol Pitch Interval Symbol Pitch Interval Symbol Pitch Interval Symbol Pitch Interval A0B2C4D5 E7F9G11HOther a1b3d6e8 f10+Up-Down

7. 7 Feature Extraction (Cont.) Generation of Significant Repeating Patterns Adapt finding repeating patterns from a music piece by considering the following constrains: 1. Maximum Length 2. Minimum Length 3. Minimum frequency

8. 8 SRP-based Classification Usefulness of SRP for Classification 1. F x,m : the frequency of the SRP x for the music m 2. Sup(x,m): the support of SRP x with respect to m 3. ASup(x,C): the aggregate support of SRP x

9. 9 SRP-based Classification (Cont.) 4. NSup(x,C): the normalized support of SRP x 5. TS(x): sum up x’s normalized support in all classes 6. PW(x,C) : the pattern weight of SRP x in class C

10. 10 SRP-based Classification (Cont.) Ex: Music piece Class SRP (Frequency) AOne I(4), II(2), IV(3) BOneI(4), III(4) CTwoI(2), V(3) DTwoV(2), VI(3) SRP (Support) Aggreagte Support I(0.45), II(0.22), IV(0.33) I(0.95), II(0.22), III(0.5), IV(0.33 I(0.5), III(0.5) I(0.4), V(0.6) I(0.4), V(1), VI(0.6) V(0.4), VI(0.6) Normalized Support Pattern Weight I(1), II(0.58) III(0.74) IV(0.64) I(0.61), II(1) III(1), IV(1) I(0.63), V(1) VI(0.75) I(0.39), V(1), VI(1) Sup(V,D) =2/(2+3) ASup(VI,Tw o) =0+0.6 NSup(V,Two) =(1-0.4+1)/(1- 0.4+1) PW(I,One) =1/(1+0.63)

11. 11 SRP-based Classification (Cont.) Similarity Measures for SRP Matching Given a Source SRP, adopt the dynamic programming approach to measure the similarity between it and each SRP in a classdynamic programming approach Assign each symbol (beat or pitch) a numerical value

12. 12 SRP-based Classification (Cont.) Ex: 1.Beat 2.Pitch Beat Symbol ABCDEFGH Value0.150.30.450.60.70.80.91.0 Pitch Symbol ABCDEFGH Value0.10.20.30.40.550.70.851.0 Pitch Symbol abdef Value0.250.350.60.750.9

13. 13 SRP-based Classification (Cont.) PS(x,y): the pattern similarity between two SRP’s x and y E(x,C): Evidence (Rhythm:melody: Where y is the target SRP of x in C

14. 14 For SRP X, the target SRP for class Two is SRP I For SRP X, the target SRP for class One is SRP III SRP-based Classification (Cont.) Ex:  similarity threshold = 0.45, the music to be classified contains two source SRP’s X and XI PS(X,I)PS(X,II)PS(X,III)PS(X,IV) 0.60.20.80.55 PS(XI,I)PS(XI,II)PS(XI,III)PS(XI,IV) 0.40.60.10.3 PS(X,I)PS(X,V)PS(X,VI) 0.60.40.5 PS(XI,I)PS(XI,V)PS(XI,VI) 0.40.50.9 Class One E(X,ONE)=0.8x0.1=0.8, E(X,TWO)=0.6x0.39=0.234

15. 15 SRP-based Classification (Cont.) Class Determination CS(C|m): classification score Ex: PS(X,I)PS(X,II)PS(X,III)PS(X,IV) 0.60.20.80.55 PS(XI,I)PS(XI,II)PS(XI,III)PS(XI,IV) 0.40.60.10.3 PS(X,I)PS(X,V)PS(X,VI) 0.60.40.5 PS(XI,I)PS(XI,V)PS(XI,VI) 0.40.50.9 Class One Class Source SRP (Frequency) Target SRP E (x,C)NSup(x,m)CS(C|m) ONEX(4)III0.8x1=0.81 0.8+0.6x0.7 5=1.25 ONEXI(2)II0.6x1=0.60.75 TWOX(4)I0.6x0.39=0.2341 0.234+0.9x 0.76=0.909 TWOXI(2)VI0.9x1=0.90.75

16. 16 Experiment Results Data Source 7 classes: Blue, Country, Dance, Jazz, Latin, Pop, Rock music Total music: 500 pieces of music Training data: 4/5 of total pieces of music Testing data: 1/5 of total pieces of music

17. 17 Experiment Results (Cont.) Impacts of Features Rhythm: set minimum freq = 3 Melody: set minimum freq = 2 Sequence length is from 4 to 6

18. 18 Experiment Results (Cont.) Impacts of Similarity Threshold

19. 19 Experiment Results (Cont.) Comparison with the HMM-based Approach

20. 20 Dynamic Programming Approach The words `computer' and `commuter' are very similar, and a change of just one letter, p->m will change the first word into the second. The word `sport' can be changed into `sort' by the deletion of the `p', or equivalently, `sort' can be changed into `sport' by the insertion of `p' The edit distance of two strings, s1 and s2, is defined as the minimum number of point mutations required to change s1 into s2, where a point mutation is one of: change a letter, insert a letter or delete a letter