1. Self-similarity and points of interest
Jasna Maver

2. What are self-similarity detectors?
Self-similarity detectors detect locations in images where local circular regions can be well approximated by average intensity values computed either for annuli or circular sectors

3. A circular region is divided either into
annuli or
circular sectors
Each annulus or circular sector is represented by the average intensity value of its pixels

4. Weighting by 1/r is used
This effectively scales all circles to have the same circumference, so all annuli are equally important
This means polar representation of a local region
The number of annuli in a local region represents scale

5. An example
Blob-like features approximate local regions by averages computed for annuli
Original image with resolution (66×72) pixels
(b) Green circles denote blob-like features obtained
for scales between 4 to 12
(c) Reconstruction obtained with the feature approximations
(d),(e) Features and their approximations

6. How are feature locations detected?
By computing
a proportional reduction error
for each pixel location and scale
Proportional reduction error is: 𝑉 𝑃 − 𝑉 𝑊 𝑉 𝑃 = 𝑉 𝐵 𝑉 𝑃
Its meaning is explained by the next slides

7. How are feature locations detected?
𝑃 consists of 𝑀×𝑁 intensity values: 𝐼 𝑚𝑛
𝑛=0,⋯,𝑁−1; 𝑚=0,⋯,𝑀−1
Sum of intensity values of 𝑛-th circular sector: 𝑅 𝑛 = 𝑚 𝐼 𝑚𝑛
Sum of intensity values of 𝑚-th annulus: 𝐶 𝑚 = 𝑛 𝐼 𝑚𝑛 𝑵 𝑃 𝑴 P
𝒏,𝒎

8. How are feature locations detected?
The total sum-of-squares 𝑉 𝑃
A local region 𝑃 of intensity values 𝐼 𝑚𝑛 is represented by the average intensity value 𝐼
This representation is evaluated by the total sum-of-squares: 𝑉 𝑃 = 𝐼 𝑚𝑛 − 𝐼 2

9. How are feature locations detected?
The between-sets sum of squares 𝑉 𝐵
A local region is represented by averages computed either for annuli or circular sectors
This representation is evaluated by the within-sets sum of squares 𝑉 𝑊
For circular sectors: 𝑉 𝑊 = 𝑛 𝑚 𝐼 𝑚𝑛 − 1 𝑀 𝑅 𝑛 2
For annuli: 𝑉 𝑊 = 𝑚 𝑛 𝐼 𝑚𝑛 − 1 𝑁 𝐶 𝑚 2
𝑉 𝐵 = 𝑉 𝑃 − 𝑉 𝑊
For circular sectors: 𝑉 𝐵 = 𝑉 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 = 1 𝑀 𝑛 𝑅 𝑛 − 𝑅 𝑅 = 1 𝑁 𝑛 𝑅 𝑛
For annuli: 𝑉 𝐵 = 𝑉 𝑟𝑎𝑑𝑖𝑎𝑙 = 1 𝑁 𝑚 𝐶 𝑚 − 𝐶 2
𝐶 = 1 𝑀 𝑚 𝐶 𝑚

10. How are feature locations detected?
A proportional reduction error (𝑉− 𝑉 𝑊 ) 𝑉 𝑃 = 𝑉 𝐵 𝑉 𝑃 tells how much of the total sum of squared errors is removed by representing a local region with the set averages, sets are here circular sectors or annuli
The maximal value 𝑉 𝐵 𝑉 𝑃 =1 means that there is no variations of intensity values within sets, variations are only between sets, representation by the set averages is error-free and sets are determined in the best way
The minimal value 𝑉 𝐵 𝑉 𝑃 =0 means that representation by the set averages is no better than representation by the average of a local region

11. How are feature locations detected?
The total sum of squares can be decomposed into three partial sums of squares:
𝑉 𝑃 = 𝑉 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 + 𝑉 𝑟𝑎𝑑𝑖𝑎𝑙 + 𝑉 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
By dividing the above equation by 𝑉 𝑃 three different proportional reduction errors or saliency measures are obtained:
1= 𝑉 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 𝑉 𝑃 + 𝑉 𝑟𝑎𝑑𝑖𝑎𝑙 𝑉 𝑃 + 𝑉 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 𝑉 𝑃
1= 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 + 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 + 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙

13. Three different types of features
𝑆_𝑟𝑎𝑑𝑖𝑎𝑙=1
(b) 𝑆_𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙=1
(c) 𝑆_𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙=1

14. How are feature locations detected?
Saliency maps are computed for different number of annuli or scales 𝑀
Image resolution: 436×228 pixels
255×(𝑆_𝑟𝑎𝑑𝑖𝑎𝑙,𝑆_𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙,𝑆_𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 ) as an RGB colour image for scale 𝑀=7, 𝑀=15, 𝑀=30
Local scale-space maxima computed on saliency maps are feature locations

15. Local scale-space maxima (LSSM)
LSSM of 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙
LSSM of 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙
LSSM of 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
120 best LSSM
400 best LSSM
120 best LSSM

16. Properties The triple (𝑆_𝑟𝑎𝑑𝑖𝑎𝑙,𝑆_𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙,𝑆_𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 )
is invariant to rotation, photometric shift, and the magnitude of the local region contrast
is covariant with translation and scaling
is robust to intra-class variations
LSSM of 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙

17. Intra-class variations
Regions belonging to the highest 30 local scale-space maxima of 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 obtained for scales between 𝑀 = 5 and 𝑀 = 40. Image resolution is approximately 140× 150 pixels. Image shows all women from the Caltech Human-Faces image set

18. Image reconstruction from LSSM of tangential saliency
Features
Feature approximations
Image
resolution 145× ×134 pixels
LSSM of
tangential saliency 2≤𝑀<26
Image reconstructed
from feature
approximations

19. Reconstructions for different scales
Resolution 145× ×134
pixels
LSSM of 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 2≤𝑀<72
LSSM of 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 12≤𝑀<72
LSSM of 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 24≤𝑀<72

20. Image reconstruction from LSSM of radial saliency
Features 3≤𝑀<72
Feature approximations
Image
resolution 145× ×134 pixels
Reconstruction from feature approximations
Features 2≤𝑀<72

21. Higher image resolution
Image reconstruction
obtained for LSSM
of 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙
Image reconstruction
obtained for LSSM
of 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙
Image resolution
292×270 pixels

22. Local region classification
𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 > 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 > 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙
𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 < 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 + 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙
𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 > 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 + 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙
𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 > 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 > 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙
𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 > 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 > 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙
𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 > 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 + 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 < 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 + 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 > 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 > 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 > 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 > 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 < 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 + 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 > 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 + 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 > 𝑆 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 > 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙
Image resolution 349×266 pixels
Scale: 𝑀=8

23. Lenna (a) (a)120 best LSSM of 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 −𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 ;
(b) 120 best LSSM
of 𝑆 𝑟𝑎𝑑𝑖𝑎𝑙 ;
(b) 1000 best LSSM
of 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙
𝑀>3
(b)
(c)

24. Lenna Image reconstruction obtained for LSSM of 𝑆 𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙
2269 features; 𝑀>3
Local region classification; 𝑀=9