1. Hiba Tariq School of Engineering
An FPGA-based implementation for Median Filter meeting the real-time requirements of Automated Visual Inspection Systems Miguel A. Vega-Rodríguez, Juan M. Sánchez-Pérez, Juan A. Gómez-Pulido
Hiba Tariq
School of Engineering

2. Outline Motivation/Background What is image processing? Classification
Parallelism
Reconfigurable computing
Experimental results
Advantages and Disadvantages
Contributions

3. Motivation
Remove high frequency or impulse noise from images while preserving quality
Producing real-time results for time-critical applications is desirable
Improving the efficiency of the algorithms is needed
Industrial automation for product classification

4. Real-time Applications
Health monitoring applications
Medical Image Processing
Factory automation
Facial recognition
Machine Vision

5. What is Image Processing?
Image processing is any form of signal processing for which the input is an image, such as a photograph or video frame and the output could be an image or a set of characteristics related to the image
The processing is usually done by computer
The data used in image processing could be in the form of pixels, features or objects

6. Types of Image Processing
Downton and Crookes classification, 1998
Low-level Image Processing (Pixels) contrast adjustment, edge detection, filtering
Intermediate-level Image Processing (Features) segmentation, feature extraction
High-level Image Processing (Objects) object classification, object recognition

7. Image Processing Pyramid
High level (objects)
Intermediate level (features)
Low level (pixels)

8. Low-level Image Processing
Focus on low-level image processing, median filtering in particular
Most low-level image processing operations are computationally intensive
Data is in the form of pixels

9. Median Filter A median filter is a spatial non-linear filter
It is known to be very robust in terms of removing high-frequency or impulse noise
It preserves the sharpness of the edges
It produces better results compared to other linear filtering techniques
The median filter operation is not reversible.

10. Median Filter Example

11. How a median filter works?
It works on a set of pixels within an area also referred to as a ‘window’ or kernel
Typically 3X3 or 5X5 kernel size is used
Pixels are first sorted within a window
Each pixel value is replaced with its median

12. Numerical Example

13. Parallelism in Image Processing
Median filtering algorithm can be made to run in parallel because of neighborhood parallelism
It is very desirable to exploit this parallelism when implementing a VLSI architecture for the filter to achieve performance gains

14. Computational Complexity
Sorting is the biggest challenge specially when the image size gets big
A general purpose processor is not enough to produce real-time results
Traditionally ASIC’s are used but they are expensive and inflexible
Reconfigurable architectures are an attractive solution

15. Proposed Architecture
Proposed a reconfigurable architecture for image processing for a visual inspection system
Compared the performance for a SW design on a GPP vs. a pure HW Implementation
Used instruction level parallelism by calculating four pixels in the same clock cycle using a 2-stage pipeline
Perform sorting in hardware using a partial sorting network

16. Sorting Network
Originally proposed by John. L. Smith of Univision Technologies Inc., Billerica, MA
The minimum exchange network required to calculate the median for nine pixels consists of 19 comparison nodes
The idea is to calculate the median for each pixel based on nine (3X3 kernel) surrounding pixels

17. Finding Median of nine pixels…
J.L Smith

18. Example: 8,7,5,2,3,1,6,4,9

19. Proposed Architecture
Presented in this paper based on J.L Smith

20. Median filter implementation
Whats happening in the registers. What kind of resource sharing is happening here?

21. Pipeline Architecture

22. Pipeline Architecture contd…
They used a Harvard architecture with 32bit buses so they were able to clock in four pixels at the same time
A two-stage pipeline was used to compute write back four pixels in one clock cycle
Pixels P1, P2 and P3 are calculated in the first stage of the pipeline and P4 in the second stage

23. Equipment Used
The software application was tested on a 350MHz Intel Pentium II Processor with 64MB RAM
The Hardware implementation consisted of:
PCI interface
HOT2-XL containing XC4062XLA FPGA
2304 CLBs
16 MHz
Filtered 30 images of size 640X480 pixels producing a throughput of 30 images per second

24. Experimental Results
SW(general purpose 350MHz Pentium 2 processor with 64MB of RAM) versus HW performance (Designed in VHDL). Execution time for operation on 30 images of 640X480 pixels with 256 gray levels.

25. Experimental Results …

26. Advantages Parallel processing is exploited using ILP
replicating filter functional units and increasing performance by four times
Speed up of 85% for performing the operation for the hardware implementation
Achieved performance by processing 30 images per second meeting real-time requirements

27. Disadvantages
Replicated four filter circuits which increased resource use by four on the FPGA
The pipelining of the architecture is not explained so well
There is no VHDL or pseudo-code provided in the paper so it will not be easy to reproduce
They didn’t mention what sorting algorithm they used in their software application
They also didn’t include any simulations of the design

28. Critiques
Two-pass median filter, weighted median filter and adaptive median filter were around in 2002.
Pixel replacement should only be performed for high probability impulse noise data to avoid image degradation

29. Contributions
Smart VLSI architecture for median filtering to produce four medians in once clock cycle
It is possible to meet real-time median filtering requirements using their approach

30. Questions?