1. ESPL 1 Motivation Problem: Amateur photographers often take low- quality pictures with digital still camera Personal use Professionals who need to document (realtors, architects) Solution: Find alternatives to picture being acquired by automating photographic composition rules Analyze scene, including detection of main subject Adapt camera settings automatically to follow rules Contribution: Automated detection of main subject Independent of indoor/outdoor setting and scene content Low implementation complexity, fixed-point computation Main subject cropped Too much background

2. ESPL 2 1: Main subject 2: Lenses 3: CCD 4: Imaging device 5: Raw data Digital Still Cameras Converts optical image to electric signal using charge coupled device Camera settings under software control Focus, e.g. auto-focus filter Zoom White balance: Corrects color distortions Shutter aperture and speed Produces JPEG compressed images

3. ESPL 3 Main Subject Detection Methods Two differently focused photographs [Aizawa, Kodama, Kubota; 1999-2002] One has foreground in focus, and other has background in focus Significant delay involved in changing the focus Bayes nets based training [Luo, Etz, Singhal, Gray; 2000-2001] Bayesian network trained on example set and tested later Training time involved: suited for offline applications Multi-level wavelet coefficients [Wang, Lee, Gray, Wiederhold; 1999-2001] Expensive to compute and analyze wavelet coefficients Iterative classification from variance maps [Won, Pyan, Gray; 2002] Optimal solution from variance maps and refinement with watershed Suitable for offline applications involving iterative passes over image

4. ESPL 4 Proposed Algorithm User starts image acquisition Focus main subject using auto-focus filter Partially blur background and acquire resulting picture Open shutter aperture (by lowering f-stop) which takes about 1 s Foreground edges stronger than background edges While acquiring user-intended picture, process blurred background picture to detect main subject Generate edge map (subtract original image from sharpened image) Apply edge detector (Canny edge detector performs well) Close boundary (e.g. gradient vector flow or proposed approximation)

5. ESPL 5 Symmetric 3 x 3 sharpening filter For integer  and , coefficients are Integer when dropping 1/(1 +  ) term Fractional when -1 – 2    and 1/(1 +  ) is power-of-two Generate edge map Subtract original image from sharpened image Main subject region now has sharper edges Generate Edge Map + + + + - f(x,y) g(x,y)f sharp (x,y) Smoothing filter f smooth (x,y) k + Model for an image sharpening filter Sharpening filter

6. ESPL 6 Boundary Closure Gradient vector flow method [Xu, Yezzi, Prince; 1998-2001] Compute gradient Outer boundary of detected sharp edges is initial contour Change shape of initial contour, depending on gradient Shape converges in approximately 5 iterations Disadvantage: computationally and memory intensive Approximate lower complexity method Select leftmost & rightmost ON pixel and make row between them ON Can detect convex regions but fails at concavities

7. ESPL 7 Implementation Complexity Number of computations and memory accesses per pixel Sharp region calculation: convolution with symmetric 3x3 filter with parameters  = 0.5 and  = 3.5; subtraction Canny edge detector: gradient computation with symmetric 3x3 filter; non-maximal suppression Digital still cameras use ~160 digital signal processor instruction cycles per pixel Processing stepMultiply-Accumulates /pixel Comparisons/ pixel Memory accesses/pixel Sharp region detection 94 Canny edge detection 924 Appr. boundary closure 22

8. ESPL 8 Results Original image with main subject(s) in focus Detected strong edges with proposed algorithm Detected main subject mask with gradient vector flow

9. ESPL 9 Conclusion Developed automated low-complexity one-pass method for main subject detection in digital still cameras Processes picture taken with blurred background Detects main subject by detecting frequency content difference between main subject and background Requires 18 multiply-accumulates, 4 comparisons, and 10 memory accesses per pixel All calculations in fixed-point arithmetic Applications: digital still cameras, surveillance, constrained image compression, and transmission and display Copies of MATLAB code, poster, and paper, available at http://www.ece.utexas.edu/~bevans/papers/2003/stillCameras

10. ESPL 10 SUBJECT DETECTION FOR DIGITAL Serene Banerjee and Brian L. Evans Embedded Signal Processing Laboratory The University of Texas at Austin AUTOMATIC MAIN

11. ESPL 11 STILL CAMERAS