1. A Framework for Photo-Quality Assessment and Enhancement based on Visual Aesthetics Subhabrata Bhattacharya Rahul Sukthankar Mubarak Shah

2. Reference http://www.cs.ucf.edu/~subh/

3. Outline Introduction Learning Aesthetics Enhancing Composition Experimental Results

4. Introduction Assessing the quality of photographs is challenging. Experienced photographers adhere to several rules of composition.  Rule of Thirds  Visual Weight Balance

5. Subject of interest is aligned to one of the stress points. Rule of Thirds

6. Rule of Thirds : Example

7. In a well composed image the visual weights of different regions satisfy the Golden Ratio. Visual Weight Balance Sea Sky k ~1.618k

8. Visual Weight Balance : Example

9. Introduction In this paper, will use these two rules to assess an image. Formulate photo quality evaluation as a machine learning problem.

10. Overview

11. Learning Aesthetics Dataset User Survey Aesthetic Features Learning and Prediction

12. Dataset Single subject Compositions (384)Landscapes/Seascapes (248)

13. User Survey 15 participants were asked to assign integer rank from 1 to 5. Each user was asked to rank no more than 30 images. Generate single ground truth for each image (F a ).

14. User Survey

15. Aesthetic Feature Extract a relative foreground position feature for images with single-foreground compositions. A visual weight ratio feature for photographs of seascapes or landscapes.

16. Defined as the normalized Euclidean distance between foreground’s mass to each four stress points. Relative foreground position

19. The ratio of the sky region, to that in the support region ( ground or sea). Visual weight ratio YgYg YkYk

22. Learning and Prediction We use SVR to learn the mappings. 150 random images for training and resting for testing.

23. Enhancing Composition Relocate the foreground object to increase the predicted appeal factor. Better balancing the visual weights of the sky and support region.

24. Why Cropping does not work? Optimal Crop

25. Semantic Segmentation Input Image Geometric Context Classifier* *D. Hoiem, A.A. Efros, and M. Hebert, "Geometric Context from a Single Image", ICCV 2005 Sky Support Post Processing Horizon Segmented Foreground

26. Optimal object placement Support Neighborhood s.t. neighbors stay “like neighbors” + Intensity Term Gradient Term

27. Example PAF = 3.22 Original Image PAF = 4.53 Optimal Solution

28. Rescaling Scaling Factor Vanishing Point Optimal location Visual Attention Center

29. Inpainting Foreground Hole Yunjun Zhang. Jiangjian Xiao. Mubarak Shah, “Region Completion in a Single Image”, EUROGRAPHICS 04 Inpaint Hole

30. Balancing visual weights YkYk YgYg Ratio of Current extents Y k +h YgYg h = vertical extent of the balanced image

31. Experimental Results PAF = 3.77 PAF = 4.25 Before After

32. Experimental Results PAF = 3.92PAF = 4.11 Before After

33. PAF = 3.98 PAF = 4.46 Experimental Results Before After

34. PAF = 4.02PAF = 4.34 BeforeAfter Experimental Results

35. PAF = 3.13 PAF = 4.19 Experimental Results BeforeAfter

36. PAF = 3.83 PAF = 4.02 Experimental Results Before After

37. PAF = 3.92 PAF = 4.38 Experimental Results Before After

38. Experimental Results PAF = 4.02 PAF = 4.71 Before After

39. Experimental Results PAF = 4.17 PAF = 4.49 Before After

40. Optimal Placement

41. Visual Weights

42. Failure case PAF = 2.34 Fa = 2.41 (Ground Truth) Before PAF = 3.63Fa = 2.54 (Ground Truth) After

43. Thank You