M-HinTS: Mimicking Humans in Texture Sorting Egon L. van den Broek Eva M. van Rikxoort Thijs Kok Theo E. Schouten Image Sciences Institute UMC Utrecht – The Netherlands

M-HinTS: Mimicking Humans in Texture Sorting Unconstrained human texture sorting continuation of previous research –180 color texture images, VisTex, OuTex, 15 categories –to be sorted into 6 groups –generated gray images from them –random order images, sequence color-gray –18 participants in controlled lab environment, monitor, lightning

M-HinTS: Mimicking Humans in Texture Sorting 180 images, VisTex and OuTex 15 categories

M-HinTS: Mimicking Humans in Texture Sorting The online M-HinTS environment –PHP –HTML 4.01 ‘layers’ –JavaScript dragging of layers 34 participants

M-HinTS: Mimicking Humans in Texture Sorting Results: human texture clustering Speed of clustering: –large order effect, second one faster –repeated measures ANOVA: gray faster (F(1,32)=11.00, p <.002). Consensus matrix per pair of participants Average consensus between all participants: color: 51% (31%-72%) gray: 52% (33%-68%) Last year’s results color: 57% gray: 56%

M-HinTS: Mimicking Humans in Texture Sorting Automatic texture clustering K-means clustering algorithm Feature vectors (Van den Broek & Van Rikxoort, 2005) –gray 32 bin histogram –4 texture features from co-occurrence matrix –11 color categories histogram –4 texture features from color correlogram color/gray, texture, combined

M-HinTS: Mimicking Humans in Texture Sorting Automatic vs. human texture clustering Consensus matrix pair overall consensus for each feature vector: –Color: 35% - 34% - 36% ( 45% - 46% - 47%) –Gray: 35% - 35% - 36% ( 44% - 45% - 42%) varying circumstances (e.g., computer, light) So, an alternative analysis …

M-HinTS: Mimicking Humans in Texture Sorting Identifying cluster strategies: Method For each participant (p) create a binary correlation matrix of 180x180 (M p ) Determine: ΣM p K means clustering: determine 6 average clusters –Distance measure: squared Euclidean distance –Random seeds, 100 iterations Use labels of the 15 categories to describe the clusters, based on the #images/category in them.

M-HinTS: Mimicking Humans in Texture Sorting Identifying cluster strategies: Results Color clusters: –1 cluster determined by semantics –1 cluster determined by low level texture features –1 garbage cluster –3 clusters unable to determine Gray scale clusters: –2 clusters determined by semantics –2 clusters determined by low level texture features –2 clusters unable to determine Conclusion: Clustering is done based on both semantics and low level features

M-HinTS: Mimicking Humans in Texture Sorting Identifying cluster strategies: Automatic clustering Color: –2 clusters are dominated by OuTex images –On average, each clusters contains 8.80 categories Gray scale: –2 clusters consist of solely OuTex images –On average, each clusters contains 8.75 categories

M-HinTS: Mimicking Humans in Texture Sorting Identifying cluster strategies: Humans versus automatic Automatic clustering isolates the OuTex textures better than humans do On average, the automatically determined clusters consist of images from more semantic categories than those of humans. This illustrates again that automatic clustering at best mimics human clustering partly.

M-HinTS: Mimicking Humans in Texture Sorting Summary Little consensus between participants –No generic human texture clustering Not able to replicate last year’s findings –not controlled monitor, lightning Participants used a combination of low level visual cues and semantics

M-HinTS: Mimicking Humans in Texture Sorting Future research subclusters in the 6 clusters sorting order or strategies of humans (z-axis) experiments same images, another number of clusters map human clusters on feature space semantic information in automatic clustering Measure monitor and lightning differences Generate much more data

The End