1. University of Zagreb, Faculty of Electrical Engineering and Computing
Automated Segmentation of Humans in Videos for Soft- and Non-Biometric De-identification
Tomislav Hrkać
University of Zagreb, Faculty of Electrical Engineering and Computing
Unska 3, Zagreb, Croatia

2. Context: de-identification
Security cameras
Privacy concerns
De-identification
Biometric
Soft biometric
Non-biometric
Important first step:
Person detection
Input: image or
video sequence
Persons
detection
Identifier
segmentation
Output: de-
identified content
De-identifying
transformation

3. Pedestrian segmentation
Output of human detector – bounding boxes around humans
More precise localisation of persons needed

4. GrabCut algorithm Semi-authomatic algorithm Minimal user input
Easily combined with background subtraction to acheive full authomatic character

5. GrabCut algorithm In theory – very good results In reality, however...
Three types of problems:
“Cutting through” at the high contrast boundary
Snapping to high contrast in the background
Enclosing concavities

6. GrabCut– how does it work?
Preliminary BG and FG specs -> GMM models
Segmentation as energy minimization: E = U + V
Minimization by graph cuts
Iterative procedure: refining BG and FG models in each step
“data term”
“smoothnes term”

7. GrabCut – details
Data term:
Smoothnes term:

8. Our modifications to GrabCut
Preliminary step: find out the rough object boundary based on background subtraction followed by morphological post-processing
Modified GrabCut:
Weighted modeling of FG and BG
Modification to the data term U to take into account preliminary classification
Modification to the smoothness term V to discourage boundaries far from the preliminary boundary

9. Modifications – details (1/3)
Weighted modeling of BG and FG
Originally each pixel initially classified as BG or FG contributes equally to BG/FG GMM models
Instead, we propose to build GMMs by weighting the contribution of each pixel based on its distance from the preliminary boundary

10. Modifications – details (2/3)
Modification of the data term
Originally:
Modified:

11. Modifications – details (3/3)
Modification of the smoothness term
Originally:
Modified:

12. Results (1/3) Qualitative evaluation
(Left –input image; middle – GrabCut; right- our method)

13. Results (2/3)
Quantitative evaluation (CDnet2014 Pedestrian dataset)

14. Results (3/3)
Quantitative evaluation (CDnet2014 Pedestrian dataset)

15. Conclusion
We proposed fully authomatic video foreground segmentation, based on GrabCut, initialized by background subtraction mask
the more relevant GMM models of foreground and background are built by weighing pixel contribution depending on their distance from the boundary of the object prior
the data and smoothness term of the energy equation are modified in a way that discourages placing boundaries far from the object prior boundary.
Evaluation on the CDNet 2014 Pedestrian Dataset:
significant improvement of recall
slight drop in precision
much better F1 measure
Further experimentation and investigation of possible improvements needed

16. Acknowledgments
This work has been supported by the Croatian Science Foundation, within the project “De-identification Methods for Soft and Non-Biometric Identifiers” (DeMSI, UIP ), and by the COST Action IC1206 “De-Identification for Privacy Protection in Multimedia Content”. This support is gratefully acknowledged