1. Minimum Barrier Salient Object Detection at 80 FPS JIANMING ZHANG, STAN SCLAROFF, ZHE LIN, XIAOHUI SHEN, BRIAN PRICE, RADOMIR MECH IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 IEEE International Conference on Computer Vision (ICCV), 2015, Santiago, Chile 1

2. Salient Object Detection Generate a Saliency Map for segmenting significant objects in an image. InputSaliency Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 2

3. Bottom-up Saliency Cues Contrast/RarityImage Boundary Connectivity Appearance Space Frequency salient IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 3

4. Previous Works Rairity/Contrast Boundary Connectivity State-of- the-art SO [Zhu et al. CVPR’14] AMC [Jiang et al. ICCV’13] GS [Wei et al. ECCV’12] RC [Cheng et al. TPAMI’15] IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 4

5. Previous Works Rairity/Contrast Boundary Connectivity Using Super-pixel? State-of- the-art SO [Zhu et al. CVPR’14]Yes AMC [Jiang et al. ICCV’13]Yes GS [Wei et al. ECCV’12]Yes RC [Cheng et al. TPAMI’15]Yes IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 4

6. Previous Works Rairity/Contrast Boundary Connectivity Using Super-pixel? State-of- the-art SO [Zhu et al. CVPR’14]Yes AMC [Jiang et al. ICCV’13]Yes GS [Wei et al. ECCV’12]Yes RC [Cheng et al. TPAMI’15]Yes Fast FT [Achanta et al CVPR’09]No HC [Cheng et al. CVPR’11]No IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 4

7. Previous Works Rairity/Contrast Boundary Connectivity Using Super-pixel? State-of- the-art SO [Zhu et al. CVPR’14]Yes AMC [Jiang et al. ICCV’13]Yes GS [Wei et al. ECCV’12]Yes RC [Cheng et al. TPAMI’15]Yes Fast FT [Achanta et al CVPR’09]No HC [Cheng et al. CVPR’11]No ProposedNo IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 4

8. Contributions FastMBD, a fast approximate Minimum Barrier Distance (MBD) [Strand et al. CVIU’13] transform algorithm, with error bound analysis. A salient object detection method based on FastMBD, which achieves state-of-the-art performance and is one order of magnitude faster! 77 FPS IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 5

9. System Overview Input L a b MBD-L MBD-a MBD-b MBD Sal-Map MBD Transform Post-processing Backgroundness IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 6

10. Measuring Boundary Connectivity by Distance Transform Compute the distance for each pixel w.r.t. the image boundary Seed Set Shortest Path IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 7

11. MBD vs Geodesic Distance In what follows, we consider a single-channel image. MBD [Strand et al. CVIU’13]Geodesic + IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 8 =

12. MBD vs Geodesic Distance MBD is robust to small pixel value fluctuation = Geodesic + IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 9 = MBD

13. FastMBD A raster-scanning iterative algorithm Share the same spirit of the raster-scanning geodesic distance transform [Toivanen, Pattern Recognition Letters, 1996] Highly efficient in practice (2ms/scan for a 320X240 image!) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 10

14. Algorithm For each visited pixel x : 1.Check each of the 4-connected neighbors IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 11 x y

15. Algorithm For each visited pixel x : 1.Check each of the 4-connected neighbors 2.Minimize the path cost 3.Update: ◦ D (x), cost of current assigned path ◦ U (x), highest value on assigned path ◦ L (x), lowest value on assigned path IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 11 D(y)D(y) U(y)U(y) L(y)L(y) D(x)D(x) U(x)U(x) L(x)L(x)

16. Some Analysis … FastMBD returns an upper-bound of the MBD for each pixel. FastMBD will eventually converge. Q: Is the converged solution of FastMBD equal to the exact MBD transform? A: No  [Falcao et al. TPAMI’04]. IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 12

17. An Error Bound Result Definition: Maximum Local Difference is the maximum absolute pixel value difference between a pair of pixels that share an edge or a corner on an image. IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 13 Measures maximal local discontinuity Under a mild condition, the approximation error of the converged solution of FastMBD is bounded by. The more continuous the image, the smaller the error.

18. Empirical Evaluation of FastMBD IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 14 PASCAL-S

19. Minimum Barrier Salient Object Detection 1.For each color channel, compute the MBD map by FastMBD 2.Post-process the averaged MBD map a)Morphological smoothing b)Multiplying a center-distance map c)Contrast enhancement IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 15 InputAveraged MBD MapOutput

20. Leveraging Backgroundness 1.For each color channel, compute the MBD map by FastMBD 2.Compute the Image Boudnary Contrast (IBC) Map 3.Post-process the averaged MBD map + IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 16 + Averaged MBD MapIBC MapOutput

21. Computing the IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 17 Image boundary region is mostly background. Compute mean color and color covariance Mahalanobis Distance to Mean Color

22. Computing the IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 17

23. Enhancement by the IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 18 W/O IBC Map With IBC Map

24. Compared Methods MB+ (full system) MB (w/oIBC map) GD (Same as MB but uses Geodesic Distance) FT[Achantaet al. CVPR’09] HC [Chengetal. CVPR’11] ] SO [Zhu etal. CVPR’14 AMC [Jiang et al. ICCV’13] HS [Yan et al. CVPR’13] GS [Wei et al. ECCV’12] RC [Cheng et al. TPAMI’15] SIA [Cheng et al. ICCV’13] Experiments IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 19 Datasets #Images Difficulty MSRA10K 10000 + DUTOmron 5168 +++ ECSSD 1000 ++ -SPASCAL 850 +++

25. Performance Precision-Recall Curve IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 20 Comparison with the State-of-the-art Comparison with Other Baselines

26. Performance Weighted F β [Margolin et al. CVPR’14] IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 21

27. 0.2 0.3 0.4 0.5 0.6 020406080100 Speed Excluding IO time Using a singe thread IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 22 FPS Mean Weighted Fb MB MB+

28. Qualitative Evaluation IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 23

29. Conclusion Propose to use MBD to measure image boundary connectivity Present FastMBD, a fast approximation MBD transform algorithm Achieve state-of-the-art performance at a substantially reduced computational cost (~80 FPS) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 24 An executable program is available on our website http://www.cs.bu.edu/groups/ivc/fastMBD/

30. Performance Area Under the Curve (AUC) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 25

31. Performance mean Average Error (mAE) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 26

32. Performance Controlling Post-processing IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 27

33. Algorithm For each visited pixel x : 1.Check each of the 4-connected neighbors 2.Minimize the path cost 3.Update D (x), L (x), U (x) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 11 D(y)D(y) U(y)U(y) L(y)L(y) D(x)D(x) U(x)U(x) L(x)L(x)