Towards automatic coin classification L.J.P. van der Maaten E.O. Postma

Introduction RICH project (Reading Images for the Cultural Heritage) Initiated by NWO-CATCH (grant 640.002.401) Institutions involved: MICC-IKAT (Maastricht University) ROB (Dutch State Service for Archaeology) People involved: E.O. Postma, A.G. Lange, H.H. Paijmans, L.J.P. van der Maaten, P.J. Boon

Introduction Automatic coin classification based on visual features May allow sorting heterogeneous coin collections, both modern and historical For modern coins, applications for charity organizations, financial institutions, and change offices (MUSCLE CIS benchmark) For historical coins, applications in the cultural heritage domain

Introduction Currently, some coin historical collections are being disclosed on the Internet, e.g., in NUMIS1 NUMIS website shows information (and sometimes photographs) on collected coins However, the use of such websites for non-experts is limited 1 A project of the Dutch Money Museum

Introduction Non-experts who find a coin would like to know what sort of coin it is i.e. coin classification based on visual features Non-experts would benefit from a system for automatic coin classification Also beneficial to experts to speed up and objectify the classification process

Introduction A modern and a historical coin photograph

Introduction This presentation Presents a number of features that can be used for the classification of modern coins Shows promising results for these features Investigates the performance of the same features on a medieval coin dataset Tries to provide some insight in why the features fail on the medieval coin data

Features Contour features Texture features Edge distance distributions Edge angle distributions Edge angle-distance distributions Texture features Gabor histograms Daubechies D4 wavelet features

Contour features Measure statistical distributions of edge pixels Edge pixels computed using Sobel filter convolution (with non-maxima suppression and dynamic thresholding) Coin borders are removed

Edge distance distributions Estimate the distribution of the distances of edge pixels to the center of the coin Rotation invariant feature Can be measured on coarse-to-fine-scales

Edge angle distributions Measure distribution of angles of edge pixels w.r.t. the baseline Not rotation invariant by definition (however, the magnitude of the Fourier transform is) Can be measured on number of fine scales

Edge angle-distance distr. Incorporate both angular and distance information in the coin stamp We measure EADD using 2, 4, 8, and 16 distance bins and 180 angular bins

Gabor histograms Convolution of coin image with Gabor filters of various scales and rotations Compute image histograms of the resulting convolution images Apply PCA for dimensionality reduction (200 dimensional)

Daubechies D4 wavelet Perform wavelet decomposition using Daubechies D4 wavelet Computed wavelet coefficients are used as features (2-, 3-, and 4-level; ahvd) Do this for 16 rotated versions of the coins in the training set (for rotation invariance) Apply PCA for dimensionality reduction (results in 200-dimensional feature vector)

Experiments Performed on the MUSCLE CIS benchmark coin dataset1 The dataset contains 692 coin classes with 2,270 coin faces Training set of 20,000 coins Test set of 5,000 coins Incorporate area measurements 1 Newer experiments than the ones described in the paper

Experiments

Experiments Classification performances (5-NN classifier) Edge distance distributions 68% Edge angle distributions 62% Edge angle-distance distributions 78% Gabor histograms 55% Daubechies D4 wavelet features 46%

Experiments Subsequently, we performed experiments on the Merovingen dataset1 Contains 4,569 early-medieval coins Class distribution skewed Experiments using 10-fold cross validation 1 Dataset property of Dutch Money Museum

Experiments Skewed class distributions Class type No. of classes Mean class size St. dev. of class size City 18 53 125 Mint master 19 69 121 Currency 4 859 1,469 Nation 12 199 438

Experiments Classification performances (naïve Bayes classifier) Feature City Mint master Currency Nation Area 16% 10% 61% 17% Edge dist. distr. 12% 8% 50% 20% Edge angle distr. 6% 34% 14% Gabor histogram 5% 25% D4 wavelet feat.

Discussion Although results on modern coin data are promising Results on Merovingen coin dataset disappointing

Discussion Reasons for results on medieval coins: Contour features highly rely on the correct estimation of the center of the stamp Texture features more suitable for coins with detailed artwork in stamps Errors and inconsistencies in these kind of datasets

Discussion Coin classified as Frankish Coin classified as Frisian

Discussion Reasons for results on medieval coins : Medieval coins have larger within-class variances due to quick deterioration of medieval coin stamps Medieval coins have smaller between-class variances (stamps often contain similar pictures, such as a cross or the head of an authority)

Discussion Reasons for results on medieval coins : Experts indicate that classifications are based on small details I.e. expert classifications are based on a large number of small (undocumented) rules Experts (consciously or not) take extrinsic information into account (such as finding location)

Discussion How should a system for automatic classification of medieval coins work? Text is highly discriminating, however, cannot be read by state-of-the-art in character recognition We foresee the development of a semi-automatic adaptive system in which the expert indicates distinguishing features of the coin Over time, the system should be able to learn the undocumented rules

Conclusions Contour and texture features perform well in the classification of modern coins The results of these features on early-medieval coins are disappointing There are various reasons why the features fail in the classification of medieval coins Future work: semi-automatic approach

Questions ?