1. Robust Segmentation of Freight Containers in Train Monitoring Videos Qing-Jie Kong*, Avinash Kumar**, Narendra Ahuja**,Yuncai Liu* **Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign *Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University, Shanghai 200240, China WACV 2009

2. Input Input : Video of an Intermodal Freight Train captured from a fixed camera with background visible before the train arrives.

3. Camera Inter-modal Train Viewing Volume Video Capture of an Intermodal Freight Train

4. Output Output : Video with background removed and thus foreground consisting of only the intermodal train. Main Application : Fast and Automatic calculation of gap lengths between consecutive containers

5. Example: Input and Output Video

6. Major Difficulties Varied Outdoor Imaging Conditions

7. Major Difficulties Different Types of Containers

8. Four Stage Coarse-to-Fine Framework Stage 1: Detecting Train Region Stage 2: Removing Background Gap Stage 3: Detecting Single Stack Stage 4: Refining Segmentation Result

9. Stage 1: Detecting Train Region Partition of the region Pixel signal in temporal domain Power spectrum of the signal

10. Stage 1: Detecting Train Region A frame in a video Frequency image of the video Histogram of the frequency image

11. Stage 1: Train Region Detection Thresholded result By the morphological operations Final result of the first stage

12. Stage 2: Removing Background Gap Background Model Background Removal Background Update

13. Background Model A background image A sub-region of the background image Histogram of the sub-region

14. Background Removal A frame in a video Result of the recognition Segmentation result

15. Background Update The background between two containers spans the complete background every some frames Splice the detected middle backgrounds to rebuild a new background image The updating calculation happens as soon as the middle background region completes the scan

16. Stage 3: Detecting Single Stack A frame in a video Result after the first two stages blobSegmentation result

17. Stage 4: Refinement of Segmentation Results Result before refinement Result after refinement Background image A window of background

18. Combination of Color Information Do all the processing in Stage 2 to the RGB channels respectively Combine the results of the three channels by the AND operation

19. Experiments Video data: 150 videos Include 1222 containers and a wide range of background conditions: – clear blue sky – bright sunlight – static heavy clouds in the day and evening – moving heavy clouds in the day and evening – rainy day (water on lens)

20. Experiments Success ratio of Stage 1: 96% Success ratio of the last three stages: Background Conditions Total Numbers of the Containers SR (Grey)SR (RGB) 1. Day/No cloud/Blue sky10485.6%96.2% 2. Day/Bright sunlight6188.9%91.8% 3. Day/Heavy clouds27898.2%100.0% 4. Day/Moving cloud12899.2%100.0% 5. Day/General situation32398.8%100.0% 6. Evening/heavy clouds98100.0% 7. Evening/Moving clouds171100.0% 8. Rainy day/Water on lens59100.0% Total122297.4%99.3%

21. Experiments

22. Operation speed – computer : Intel(R) Core(TM)2 Due CPU 2.53-GHz processor and 3.2-GB MHz RAM. – average processing speed: 4 frames per second (fps)

23. Conclusion The proposed method – combines the information in frequency and spatial domain – is robust to varieties of background conditions – can employ videos from un-calibrated cameras – is being integrated into a real time vision system for intelligent train monitoring

24. Thanks to BNSF