1. Fast Census Transform-based Stereo Algorithm using SSE2
Young Ki Baik*
Kyoung Mu Lee
Computer Vision Lab.
School of Electrical Engineering and Computer Science
Seoul National University

2. Contents Stereo Vision Census Transform Stereo Vision Fast approaches
Experimental result
Conclusion and Future work
These are contents what I’ll present you.
At first, I’ll introduce stereo vision and census transform stereo vision.
After that, I’ll show you fast approaches for stereo vision and experimental result.
Finally, I’ll finish my presentation with conclusion and future work.

3. Introduction What is the stereo vision?
The stereo vision is the method to extract 3D information using image from different view points.
Topographical survey
Obstacle detection
Object tracking
Face recognition
What is the stereo vision?
Stereo vision is the method to extract 3D information using image from different view points.
And It can be applied to Topographical survey, obstacle detection, object tracking, face recognition and so on.

4. Introduction Trade off of algorithms Algorithm for accurate results
Complex computation and iteration
Slow processing time
Unable to realize real-time system
Algorithm for fast processing time
Simple computation and no iteration
Fast processing time
Unable to realize accurate system
Stereo vision has the trade off.
For accurate results, algorithm has complex computation and iteration, slow processing time.
For these reason, algorithms are unable to realize real-time system.
For fast processing time, algorithm has simple computation and no iteration, fast processing time.
But it is unable to realize accurate system.
Our approach, we will focus on fast processing algorithm for stereo vision.

5. Introduction Fast stereo vision algorithm Window size invariant method
Box filtering method
“Box-filtering techniques”, M.J.McDonnell (CGIP-81’)
“Real time correlation-based stereo : algorithm, implementations and applications”, Olivier Faugeras , Zhengyou Zhang , … (Tech.Rep.RR-2013, INRIA,1993)
Disparity range invariant method
Rectangular subregioning method
“Rectangular Subregioning and 3-D Maximum-Surface Techniques for Fast Stereo Matching”, Changming Sun (CVPR-2001)
Parallel processing technique
For the fast stereo vision algorithm, these 3 items can be considered.
At first, window size invariant method or box filtering method was introduced from these references.
Also, disparity range invariant method was introduced by Changming sun.
Finally, parallel processing technique is used for fast stereo vision.

6. Introduction Problem Census transform
Real images from grabbers can not assure of brightness consistency in corresponding region.
Intensity correlation method is not proper for real images.
Census transform
Census transform has been evaluated as the method robust to radiometric distortion.
J. Banks and P. Corke, "Quantitative evaluation of matching methods and validity measures for stereo vision," Int. J. Robotics Research, vol. 20, pp , July 2001.
Heiko Hirschmller, "Improvements in Real-Time Correlation Based Stereo Vision", Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision, pp , Kauai, Hawaii, December 2001.
In common stereo vision algorithm, it does not consider following problem.
The problem is that Real images from grabbers can not assure of brightness consistency in corresponding region.
Therefore Intensity correlation method is not proper for real images.
Generally, LOG filtering method are used for solving these problem.
But, census transform has been evaluated as the method robust to radiometric distortion from these papers.
So, we use census transform method in order to implement more accurate and fast stereo vision algorithm.

7. Census Transform Stereo Vision
Census transform converts relative intensity difference to 0 or 1 and deforms 1 dimensional vector as much as window size of census transform.
210
159 99 86 39
198
170 32 67 47 45 78
102 98 30 40 33
115
109 31 26
130
121 1 X
Census transform window (CTW)
I’ll talk about census transform.
Census transform converts relative intensity difference to 0 to 1 and deforms 1 dimensional vector as much as window size of census transform.
And vector will be multiple proportion of 8, therefore we can use parallel processing effectively.

8. Census Transform Stereo Vision
Result of census transform
Census transform makes data of (image size * vector size).
(Square size of CTW)-1
Height
Width
Height
And, census transform makes data of image size by vector size.
With this result…
Width

9. Census Transform Stereo Vision
Sum of Hamming distance
The Hamming distance of two transformed vectors with correlation windows is used to find corresponding region.
Sum of Hamming distance
Disparity range
The Hamming distance of two transformed vectors with correlation windows is used to find
corresponding region.
And we can obtain 3d information from this 3d space.
This is census transform based stereo vision.
Right census
transformed vector
Left census
transformed vector
3D disparity space

10. Census Transform Stereo Vision
Complexity of algorithm
Census transform-based stereo vision (CTSV) has high complexity.
N : Searching window size
D : Disparity range
C : Census transform window size
Method
Complexity
Conventional
stereo vision ND
Census transform
CND
But census transform-based stereo vision has high complexity.
As you can see in this table, complexity of census transform-based stereo vision is CND where
N is searching window size.
D is disparity range
C is Census transform window size
In order to overcome this slow processing time, we try to apply fast approaches to CTSV.

11. Fast Census Transform Stereo Vision
Hamming distance
Parallel processing
- SSE2
8bit look-up table
Parallel processing
- SSE2
Correlation
fast census transform-based stereo vision consists of these 3 parts.
We use parallel processing for census transform, 8-bit look-up table or parallel processing for computing Hamming distance,
Moving window technique and parallel processing for correlation.
Moving window technique
Parallel processing
- SSE2

12. Fast Census Transform Stereo Vision
SSE2 (Streaming SIMD Extension 2)
128-bit SIMD packed integer & floating point arithmetic operation
Cache and memory management operation
Continuous memory structure is required
No advantages in separate data
SSE2 offers 128-bit SIMD packed integer and floating point arithmetic operation and
It provides cache and memory management operation.
In order to use SSE2, continuous memory structure is required and there are no advantages in separate data.

13. Fast Census Transform Stereo Vision
Fast approaches (census transform)
Usage of parallel processing (SSE2)
16 pixels are loaded to XMM(SSE2 memory) and computed at once.
For the census transform, we use SSE2.
In this process, 16 pixels are loaded to XMM and computed at once.

14. Fast Census Transform Stereo Vision
Fast approaches (sum of Hamming distance)
Usage of 8bit look-up table (LUT)
Parallel processing : SSE2
Parallel processing is faster than 8 bit LUT
For computing sum of Hamming distance, we use 8 bit look up table or parallel processing.
And, we can verify that parallel processing is faster than 8 bit look up table.

15. Fast Census Transform Stereo Vision
Fast approaches (correlation)
Moving window technique
For correlation, we use moving window technique which considers near memory access with box filtering.

16. Fast Census Transform Stereo Vision
Fast approaches (correlation)
Combination of moving window and SIMD
Moving window technique for x, y-axis
SSE2 for d-axis
In order to make faster approach for correlation,
We combine moving window for x, y-axis and SSE2 for d-axis.

17. Experimental result Environment Condition System : Pentium-IV 2.4GHz
Cache memory : 512Kbyte
Camera : Stereo Mega-D (Videre design)
Condition
Image size : 320 x 240 gray stereo images
Census transform window size :5x5, 7x7, 9x9
Disparity searching range : 32
Correlation window size : 11x11
I’ll show you experimental result.
We use pentinu-4 2.4GHz and 512Kbyte cache memory, and stereo mega-d camera from videre design.
We use 320x240 gray stereo images.
And we test alternative census transform window size.
Disparity searching range is 32.
And correlation window size is 11x11.

18. Experimental result Detail processing time
32 disparity searching range
Method
Census
Transform
Window
Time (msec)
Frame
Rate CT
Cal HD
Corr
Total
with
SSE2 5
5.9
17.5
5.8
29.2
34.2 7
11.6
29.6
5.6
46.8
21.4 9
35.4
52.6
4.3
92.3
10.8
without
34.3
24.1
14.2
72.5
13.8
66.2
35.1
14.1
115.4
8.7
208.3
66.5
289.0
3.5
For the census transform-based stereo vision with SSE2, we can get 34.2 frame rate.

19. Experimental result Performance of census transform stereo vision And
This is the graph of performance for census transform stereo vision

20. Conclusion and Future work
Moving window technique reduces processing time to constant except in transforming stage
SSE2 instructions reduces running time by 2.5 to 3 times
Possibility for faster result
Specialized Instruction
16 bit look-up table
Fixed window size of census transform
Future work
Applying real-time approach to another stereo algorithm
Combine stereo system to other applications
We can verify that moving window technique reduces processing them to constant except in transforming stage
And sse2 instructions reduces running time by 2.5 to 3 times.
We can expect faster results with specialized instruction in sse2, 16 bit look-up table and fixed census transform window size.
For future works, we will apply real-time approaches to another stereo algorithm and combine stereo system to other applications.
This is end of my talk. Thank you for your attention.
Is there any question?