2. What is Image Quality Assessment?
Image quality is a characteristic of an image that measures the perceived image degradation
It plays an important role in various image processing application.
Goal of image quality assessment is to supply quality metrics that can predict perceived image quality automatically.
Two Types of image quality assessment
Objective quality assessment
Subjective quality assessment

3. Two Types of image quality assessment
Objective (MSE, PSNR, SSIM) [1,2]
Automated
Low cost
Less accurate
Subjective: Mean Opinion Score (MOS) [4]
Involving human participants
Expensive
More accurate and ecologically valid

4. Subjective Quality Measure
The best way to find the quality of an image is
to look at it
because human eyes are the ultimate viewer.
Subjective image quality is concerned with how image is perceived by a viewer and give his or her opinion on a particular image.

5. Subjective Quality Measure
The mean opinion score (MOS) has been used for subjective quality assessment for many years.
In Audio standard subjective test listeners (both male and female, professional and non-professional) rate the heard audio quality over the communication medium being tested.
In case of image, the image quality is measured by viewers (both male and female, professional and non-professional) on the display devise or printed media.
However, this approach is too inconvenient, time consuming and expensive.

6. Example of mean opinion score (MOS)
The MOS is generated by averaging the result of a set of standard, subjective tests.
MOS is an indicator of the perceived image quality.
Mean Opinion Score of 1 is worst image quality and 5 is best [4].
Mean Opinion Score (MOS)
MOS
Quality
Impairment 5
Excellent
Imperceptible 4
Good
Perceptible but not annoying 3
Fair
Slightly annoying 2
Poor
Annoying 1
Bad
Very annoying

7. Mean Opinion Score (MOS) Subjective test

8. Objective Quality Measure

9. Objective Quality Measure
Mathematical models that approximate the results of subjective quality assessment.
Goal of objective evaluation is to develop quantitative measure that can predict perceived image quality.

10. Objective Quality Measure
Objective quality measure plays variety of roles:
To monitor and control image quality for quality control systems
To benchmark image processing systems;
To optimize algorithms and parameters;
To help home users better manage their digital photos and evaluate their expertise in photographing.

11. Objective evaluation Three types of objective evaluation
It is classified according to the availability of an original image with which distorted image is to be compared
Full reference (FR)
No reference –Blind (NR)
Reduced reference (RR)

12. Full reference quality metrics
MSE and PSNR: the most widely used video/image quality metrics during last 20 years.
SSIM: new metric (was suggested in 2004) shows better results than PSNR with reasonable computational complexity increasing.
A great effort has been made to develop new objective quality measures for image/video that incorporate perceptual quality measures by considering the human visual system (HVS) characteristics.

13. HVS – Human visual system
Quality assessment (QA) algorithms predict visual quality by comparing a distorted signal against a reference, typically by modeling the human visual system.
The objective image quality assessment is based on well defined mathematical models that can predict perceived image quality between a distorted image and a reference image.
These measurement methods consider human visual system (HVS) characteristics in an attempt to incorporate perceptual quality measures.

14. MSE – Mean square error MSE and PSNR are defined as
Where x is the original image and y is the distorted image.
M and N are the width and height of an image.
L is the dynamic range of the pixel values.
If the MSE decrease to zero, the pixel-by-pixel matching of the
images becomes perfect.

15. Same MSE but different qualities
MSE for all images = 215

16. SSIM – Structural similarity index
Recently proposed approach for image quality assessment.
A metric for measuring the similarity between two images.
Its value lies between [0,1]
The SSIM is designed to improve on traditional metrics like PSNR and MSE, which have proved to be inconsistent with human eye perception. Based on human visual system.

17. SSIM measurement system

18. How to measure SSIM
Assume we have two images x and y, where xi and yi ,
i = 1, 2, …, N are their elements.
The aim is to determine how similar images x and y are.
Mean and SD for image x is defined as follows:

19. How to measure SSIM To compare the similarity of two images x and y,
we need to compare their
Luminance : l
Contrast: c and
Structure: s
By combining them in a weighted function as follows: S(x,y) = f ( l(x,y), c(x,y), s(x,y) )

20. How to measure SSIM S(x,y) = f ( l(x,y), c(x,y), s(x,y) )
So we need to define the above three functions l,c,s and also the function f(.).
However, the similarity measure S(x,y) should satisfy the following conditions:
1- Symmetry: S(x,y) = S(y,x)
2- Boundedness: S(x,y) ≤ 1
3- Unique maximum: S(x,y) = 1 if and only if x=y, that is
xi = yi, for all i=1,2, …, N)

21. How to measure SSIM S(x,y) = f ( l(x,y), c(x,y), s(x,y) )
Luminance comparison is defined as:
Where the constant C1 to avoid division by a very small (or zero) number.
𝐶 1 =( 𝐾 1 𝐿) 2
Where L is the dynamic range of pixel values (255 for 8-bit gray scale image), and K1 « 1 is a small constant.
𝑙 𝑥,𝑦 = 2 𝜇 𝑥 𝜇 𝑦 + 𝐶 1 𝜇 𝑥 2 + 𝜇 𝑦 2 + 𝐶 1

22. How to measure SSIM S(x,y) = f ( l(x,y), c(x,y), s(x,y) )
Contrast comparison is defined as:
Where the constant C1 to avoid division by a very small (or zero) number.
𝐶 2 =( 𝐾 2 𝐿) 2
Where L is the dynamic range of pixel values (255 for 8-bit gray scale image), and K2 « 1 is a small constant.
𝑐 𝑐,𝑦 = 2 𝜎 𝑥 𝜎 𝑦 + 𝐶 2 𝜎 𝑥 2 + 𝜎 𝑦 2 + 𝐶 2

23. How to measure SSIM S(x,y) = f ( l(x,y), c(x,y), s(x,y) )
Structure comparison is defined as:
Where the constant C3 to avoid division by a very small (or zero) number.
𝐶 3 =( 𝐶 2 /2)
𝑠 𝑥,𝑦 = 𝜎 𝑥𝑦 + 𝐶 3 𝜎 𝑥 𝜎 𝑦 + 𝐶 3
𝑆𝑆𝐼𝑀 𝑥,𝑦 = 𝑙(𝑥,𝑦) 𝛼 𝑐(𝑥,𝑦) 𝛽 𝑠(𝑥,𝑦) 𝛾

24. How to measure SSIM
Where α > 0, β > 0, γ > 0 are parameters used to adjust the relative importance of the components.
It is easy to verify that this definition of SSIM satisfies the three conditions:
1- Symmetry: S(x,y) = S(y,x)
2- Boundedness: S(x,y) ≤ 1
3- Unique maximum: S(x,y) = 1 if and only if x=y,
𝑆𝑆𝐼𝑀 𝑥,𝑦 = 𝑙(𝑥,𝑦) 𝛼 𝑐(𝑥,𝑦) 𝛽 𝑠(𝑥,𝑦) 𝛾

25. How to measure SSIM
To simplify the above equation, we set α = β = γ = 1 SSIM ε [0,1], Values closer to 1 indicate higher similarity.
𝑆𝑆𝐼𝑀 𝑥,𝑦 = 𝑙(𝑥,𝑦) 𝛼 𝑐(𝑥,𝑦) 𝛽 𝑠(𝑥,𝑦) 𝛾
𝑆𝑆𝐼𝑀 𝑥,𝑦 = (2 𝜇 𝑥 𝜇 𝑦 + 𝐶 1 )(2 𝜎 𝑥𝑦 + 𝐶 2 ) ( 𝜇 𝑥 2 +𝜇 𝑦 2 + 𝐶 1 )( 𝜎 𝑥 2 + 𝜎 𝑦 2 + 𝐶 2 )

26. How to measure SSIM
If we divide images x and y into blocks and then compute the similarity of each two corresponding blocks, and give a weight to the similarity measure of each block, then the average SSIM over all blocks is a new measure called
MSSIM
Where M is the number of blocks, and xi and yj are two corresponding image blocks.
𝑆𝑆𝐼𝑀 𝑥,𝑦 = (2 𝜇 𝑥 𝜇 𝑦 + 𝐶 1 )(2 𝜎 𝑥𝑦 + 𝐶 2 ) ( 𝜇 𝑥 2 +𝜇 𝑦 2 + 𝐶 1 )( 𝜎 𝑥 2 + 𝜎 𝑦 2 + 𝐶 2 )

27. MSE VS MSSIM original Image MSE=0, MSSIM=1 MSE=225, MSSIM=0.949

28. MSE vs. MSSIM
Distorted image: Salt & pepper Noise
Distorted image: Spackle Noise
Distorted image: Gaussian Noise
Distorted image:Blurred
Distorted image: JPEG compressed
Distorted image: Contrast Stretch
Original image
Sparkle noise causes artificial artifacts in digital images.
It appears as a bright spots which have a typical intensity of 40% of maximum intensity.

29. MSE vs. MSSIM simulation result
Type of Noise
MSE
MSSIM
Salt & Pepper Noise
228.34
0.7237
Spackle Noise
225.91
0.4992
Gaussian Noise
226.80
0.4489
Blurred
225.80
0.7136
JPEG compressed
213.55
0.3732
Contrast Stretch
406.87
0.9100

30. MSE vs. MSSIM
MSE= MSSIM = MSE = MSSIM =0.4992

31. MSE vs. MSSIM
MSE = MSSIM = MSE = MSSIM =0.7136

32. MSE vs. MSSIM
MSE = MSSIM = MSE = MSSIM =0.910

33. Why MSE is poor?
MSE and PSNR are widely used because they are simple and easy to calculate and mathematically easy to deal with for optimization purpose.
There are a number of reasons why MSE or PSNR may not correlate well with the human perception of quality.

34. Why MSE is poor?
Digital pixel values, on which the MSE is typically computed,
may not exactly represent the light stimulus entering the eye.
Simple error summation, like the one implemented in the MSE formulation, may be markedly different from the way the HVS and the brain arrives at an assessment of the perceived distortion.
Two distorted image signals with the same amount of error energy may have very different structure of errors, and hence different perceptual quality.

35. Conclusion
We consider the proposed SSIM indexing approach as a particular implementation of the philosophy of structural similarity, from an image formation point of view.

36. References
[1] Z. Wang and A. C. Bovik, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Processing, vol. 13, pp. 600 – 612, Apr
[2] Z. Wang and A. C. Bovik, “Modern image quality assessment”, Morgan & Claypool Publishers, Jan
[3] X. Shang, “Structural similarity based image quality assessment: pooling strategies and applications to image compression and digit recognition” M.S. Thesis, EE Department, The University of Texas at Arlington, Aug  
[4] ITU-R Rec. BT : Methodology for the Subjective Assessment of Quality for Television Pictures. June 2002.