1. Image Processing IP cores
© eVS Srl, May 30, 2005

2. Vision, Image Processing and Sound Lab
Cooperation
University of Verona
Department of Computer Science
Ultimodule
Vision, Image Processing and Sound Lab
Confidential © 2005 eVS Srl.

3. Research Areas Image Processing Computer Vision Pattern Recognition
Image Restoration
Feature Extraction (Edge, Corner, Region, Gradient)
Computer Vision
Mosaics and Video summarization
Model-Based Tracking
Stereopsis and 3D reconstruction
Pattern Recognition
Statistical techniques
Clustering and classifications of sequences
Object Recognition
Confidential © 2005 eVS Srl.

4. Applications Security and Monitoring Industrial Marine Automotive
Video surveillance
Intrusion detection
Industrial
Active video surveillance
Robotic arm motion
Measurement of points and distances
Marine
Auto docking
Support to navigation (stabilization)
Hazard detection
Automotive
Active security
Autonomous guided vehicles
Confidential © 2005 eVS Srl.

5. Sensing The Real World Stereo sensor to reconstruct the 3rd dimension
Robustness to brightness changes on scene
Real time performance
Output Image
robust RGBZ
resolution up to 512x512
up to 100 fps
Confidential © 2005 eVS Srl.

6. Image Processing Engine
The image processing engine implements most of the algorithms required when an application is asked to rely on vision to make decisions and control
Real time performance
High computational power
Self-contained modules
Compact configuration
Common interface
High modularity for different contexts
Optical (2D) and range (3D) image support
Confidential © 2005 eVS Srl.

7. Stereopsis 1
The computational stereopsis is the process used to obtain the depth information from a pair of images coming from two cameras that see the same scene from two different points of view.
We can distinguish two sub-problems:
Corresponding points detection consists in finding which points in the left and right images are projection of the same point of the scene. The images must be rectified in order to ensure that corresponding points belong to the same horizontal line. Rectification reduces the algorithm complexity.
3D reconstruction: once the correspondences, the relative position of cameras, and the internal sensor parameters are known, the 3D position in the scene can be calculated for each point of the two 2D images.
Confidential © 2005 eVS Srl.

8. Stereopsis 2 Left view Right view Disparity Disparity hot = close
brigh = close
Confidential © 2005 eVS Srl.

9. Left image rectification Right image rectification
RGBZ sensor
Stereo
Post processing
Stereo matching
Left image rectification
Z reconstruction
RGBZ image
Right image rectification
Confidential © 2005 eVS Srl.

10. Image Processing Engine: a possible configuration
Project Overview
Image Processing Engine:
a possible configuration
Post-
processing
Corner
RGBZ
Sensor
Motion estimation
Filtering
Gradient
Edge
Warping
Segmentation
Confidential © 2005 eVS Srl.

11. Filtering
The noise is reduced by convolving the intensity image with a gaussian kernel (linear filtering)
The amount of filtering can be controlled by changing the coefficients of the convolution mask
Filtering
Noisy image (Gaussian noise σ = 0.01)
Filtered image (5x5 Gaussian filter σ = 1)
Confidential © 2005 eVS Srl.

12. Segmentation
Segmentation is usually one of the first steps in image analysis
The purpose of image segmentation is to subdivide an image into meaningful, non-overlapping regions
Our implementation produces as a result a binary image which distinguishes between background and foreground
threshold
Input image
Histogram
Segmented image
Confidential © 2005 eVS Srl.

13. Edge detection
An edge is a set of connected pixels that lie on the boundary between two regions
To detect an edge we apply a threshold to the magnitude of image gradient
Gradient is computed by convolving the image with a 3x3 Prewitt mask, and edges are thinned applying non-maximum suppression
Edge
Detection
Original image
Edge
Confidential © 2005 eVS Srl.

14. Corner detection
Corner points are detected by a significant change of the gradient values along two directions
The core receives in input the image gradients and the required number of corners, and outputs the coordinates and the degree of confidence of the extracted corners.
Corners are easy to track
Once the position of corners is known along the video sequence, many informations on camera motion can be retrived.
Confidential © 2005 eVS Srl.

15. Warping
Warping consists in applying a geometrical spatial transformation T:R2→R2 to the image coordinates, and re-sampling the grid
Some features of our image warping module:
affine transformation
backward mapping
bilinear interpolation
Confidential © 2005 eVS Srl.

16. Summary 1 SPACE Slices BRAM DSP48s
Filtering
184 2 3
Edge-detection
1195 5
Corner-detection
1799 14 8
Warping
422 4
Segmentation
858
Rectification
1312
Stereo
n.a.
Performances estimation on Xilinx Virtex4 LX60 with 8 bit depth 512x512 input images
Confidential © 2005 eVS Srl.

17. Summary 2 TIME Max Freq (MHz) Pixel Rate (MHz) Frame rate (fps)
Filtering
197 49
200
Edge-detection 53 9 36
Corner-detection 42 21 86
Warping 10 26
105
Segmentation 91
370
Rectification
209 52
212
Stereo
n.a.
Performances estimation on Xilinx Virtex4 LX60 with 8 bit depth 512x512 input images
Confidential © 2005 eVS Srl.

18. Work in progress
Motion estimation: frame to frame geometric image transformation computation
Video stabilization: unwanted motion compensation
2D Feature Tracker: correlation-based tracker of a region of interest
Hough Transform: straight lines detection
Confidential © 2005 eVS Srl.

19. Development tools Matlab: Xilinx System Generator for DSP 7.1:
Algorithm design
Prototyping
High level verification
Xilinx System Generator for DSP 7.1:
Hardware design
Simulation
Hardware co-simulation (via JTAG)
Initial resources estimation
Xilinx ISE 7.1:
Implementation
Integration
Mentor ModelSim SE 6.0:
Confidential © 2005 eVS Srl.

20. Development boards 3 Xilinx Spartan3 Starter Kit Boards 1 ML310
200,000 gates Xilinx Spartan3 XC3S200FT256 FPGA
2Mbit Xilinx XCF02S Platform Flash
1M-byte of Fast Asynchronous SRAM
1 ML310
Virtex-II Pro XC2VP30
256MB DDR memory
512MB CompactFlash card
multiple PCI slots
1 SCM40
NEC VR4131 MIPS
DDR SDRAM
200,000 gates Spartan2
Confidential © 2005 eVS Srl.

21. Contacts eVS is: Prof. Vittorio Murino: vittorio.murino@evsys.net
Dott. Roberto Marzotto:
Dott. Alessandro Negrente:
Dott. Marco Monguzzi:
Phone / fax:
Web:
Confidential © 2005 eVS Srl.