Hiba Tariq School of Engineering An FPGA-based implementation for Median Filter meeting the real-time requirements of Automated Visual Inspection Systems Miguel A. Vega-Rodríguez, Juan M. Sánchez-Pérez, Juan A. Gómez-Pulido Hiba Tariq School of Engineering

Outline Motivation/Background What is image processing? Classification Parallelism Reconfigurable computing Experimental results Advantages and Disadvantages Contributions

Motivation Remove high frequency or impulse noise from images while preserving quality Producing real-time results for time-critical applications is desirable Improving the efficiency of the algorithms is needed Industrial automation for product classification

Real-time Applications Health monitoring applications Medical Image Processing Factory automation Facial recognition Machine Vision

What is Image Processing? Image processing is any form of signal processing for which the input is an image, such as a photograph or video frame and the output could be an image or a set of characteristics related to the image The processing is usually done by computer The data used in image processing could be in the form of pixels, features or objects

Types of Image Processing Downton and Crookes classification, 1998 Low-level Image Processing (Pixels) contrast adjustment, edge detection, filtering Intermediate-level Image Processing (Features) segmentation, feature extraction High-level Image Processing (Objects) object classification, object recognition

Image Processing Pyramid High level (objects) Intermediate level (features) Low level (pixels)

Low-level Image Processing Focus on low-level image processing, median filtering in particular Most low-level image processing operations are computationally intensive Data is in the form of pixels

Median Filter A median filter is a spatial non-linear filter It is known to be very robust in terms of removing high-frequency or impulse noise It preserves the sharpness of the edges It produces better results compared to other linear filtering techniques The median filter operation is not reversible.

Median Filter Example

How a median filter works? It works on a set of pixels within an area also referred to as a ‘window’ or kernel Typically 3X3 or 5X5 kernel size is used Pixels are first sorted within a window Each pixel value is replaced with its median

Numerical Example

Parallelism in Image Processing Median filtering algorithm can be made to run in parallel because of neighborhood parallelism It is very desirable to exploit this parallelism when implementing a VLSI architecture for the filter to achieve performance gains

Computational Complexity Sorting is the biggest challenge specially when the image size gets big A general purpose processor is not enough to produce real-time results Traditionally ASIC’s are used but they are expensive and inflexible Reconfigurable architectures are an attractive solution

Proposed Architecture Proposed a reconfigurable architecture for image processing for a visual inspection system Compared the performance for a SW design on a GPP vs. a pure HW Implementation Used instruction level parallelism by calculating four pixels in the same clock cycle using a 2-stage pipeline Perform sorting in hardware using a partial sorting network

Sorting Network Originally proposed by John. L. Smith of Univision Technologies Inc., Billerica, MA The minimum exchange network required to calculate the median for nine pixels consists of 19 comparison nodes The idea is to calculate the median for each pixel based on nine (3X3 kernel) surrounding pixels

Finding Median of nine pixels… J.L Smith

Example: 8,7,5,2,3,1,6,4,9

Proposed Architecture Presented in this paper based on J.L Smith

Median filter implementation Whats happening in the registers. What kind of resource sharing is happening here?

Pipeline Architecture

Pipeline Architecture contd… They used a Harvard architecture with 32bit buses so they were able to clock in four pixels at the same time A two-stage pipeline was used to compute write back four pixels in one clock cycle Pixels P1, P2 and P3 are calculated in the first stage of the pipeline and P4 in the second stage

Equipment Used The software application was tested on a 350MHz Intel Pentium II Processor with 64MB RAM The Hardware implementation consisted of: PCI interface HOT2-XL containing XC4062XLA FPGA 2304 CLBs 16 MHz Filtered 30 images of size 640X480 pixels producing a throughput of 30 images per second

Experimental Results SW(general purpose 350MHz Pentium 2 processor with 64MB of RAM) versus HW performance (Designed in VHDL). Execution time for operation on 30 images of 640X480 pixels with 256 gray levels.

Experimental Results …

Advantages Parallel processing is exploited using ILP replicating filter functional units and increasing performance by four times Speed up of 85% for performing the operation for the hardware implementation Achieved performance by processing 30 images per second meeting real-time requirements

Disadvantages Replicated four filter circuits which increased resource use by four on the FPGA The pipelining of the architecture is not explained so well There is no VHDL or pseudo-code provided in the paper so it will not be easy to reproduce They didn’t mention what sorting algorithm they used in their software application They also didn’t include any simulations of the design

Critiques Two-pass median filter, weighted median filter and adaptive median filter were around in 2002. Pixel replacement should only be performed for high probability impulse noise data to avoid image degradation

Contributions Smart VLSI architecture for median filtering to produce four medians in once clock cycle It is possible to meet real-time median filtering requirements using their approach

Questions?