Multiprocessor Architecture for Image processing Mayank Kumar – 2006EE10331 Pushpendre Rastogi – 2006EE50412 Under the guidance of Dr.Anshul Kumar

Objectives To learn to work on FPGA platform. To learn to work on FPGA platform. Learning the design philosophy of a soft core multiprocessor architecture. Learning the design philosophy of a soft core multiprocessor architecture. Using multiprocessor architecture to implement adaptive background mixture model for motion segmentation. i.e Background modeling and change detection algorithm. (Cris stauffer) Using multiprocessor architecture to implement adaptive background mixture model for motion segmentation. i.e Background modeling and change detection algorithm. (Cris stauffer)

Motivation Signal processing, especially Image processing involves repetitive computation which can easily be divided into parallel computations Signal processing, especially Image processing involves repetitive computation which can easily be divided into parallel computations There are many algorithm that follows locally sequential globally parallel computation. There are many algorithm that follows locally sequential globally parallel computation. Surveillance camera related processing need economic real time solutions. Surveillance camera related processing need economic real time solutions. FPGA provides an easy way to alter design depending on algorithm requirements – making soft multicore processing feasible. FPGA provides an easy way to alter design depending on algorithm requirements – making soft multicore processing feasible. Architecture of the Processing Elements (PE) could be optimized for image processing. Architecture of the Processing Elements (PE) could be optimized for image processing.

Adaptive background mixture model The algorithm helps to generate an adaptive model for the background and helps in motion detection. The algorithm helps to generate an adaptive model for the background and helps in motion detection. This can be applied for surveillance. This can be applied for surveillance. The algorithm is spatially parallel, thus computations for different part of the image can be done simultaneously. The algorithm is spatially parallel, thus computations for different part of the image can be done simultaneously. This will help in real time operation of this algorithm on embedded platform. This will help in real time operation of this algorithm on embedded platform.

Nature of parallelism Data Partitioning: The task is partitioned so that each processor performs exactly the same function, but on different sub-blocks of data. Data Partitioning: The task is partitioned so that each processor performs exactly the same function, but on different sub-blocks of data. For different image regions, our chosen algorithm is sequential in nature, which can be efficiently implemented through a processor. For different image regions, our chosen algorithm is sequential in nature, which can be efficiently implemented through a processor.

Basic Idea Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 MEMORYMEMORY Video DAC MPMC Monitor Array Topology

Basic Idea Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 MEMORYMEMORY Video DAC MPMC Monitor Array Topology

Inter-processor Communication [12] For transferring large chunks of image data, we will be using shared external DDR Ram as For transferring large chunks of image data, we will be using shared external DDR Ram as It provides large memory space for storing multiple frames. It provides large memory space for storing multiple frames. Multiple processors can access the RAM simultaneously using MPMC. Multiple processors can access the RAM simultaneously using MPMC. For sharing intermediate computation results we will use FSL Links between neighboring processors. For sharing intermediate computation results we will use FSL Links between neighboring processors. Unidirectional point-to-point communication. Unidirectional point-to-point communication. Unshared non-arbitrated communication mechanism. Unshared non-arbitrated communication mechanism. FIFO based communication. FIFO based communication.

Network Topology [1] Completely meshed: Each node is connected to all other nodes. Adv: Reduce inter processor communication time. Disadvantage: Max 9 processors possible. Completely meshed: Each node is connected to all other nodes. Adv: Reduce inter processor communication time. Disadvantage: Max 9 processors possible. Ring Network: Ring Network: Star Network Star Network Array Network: (our choice) Array Network: (our choice)

Overall Plan Analysis of the algorithm. Analysis of the algorithm. Configuring Video input and output for XUPV2P FPGA kit. Configuring Video input and output for XUPV2P FPGA kit. Finalizing the architecture Finalizing the architecture For one processor For one processor For two processors For two processors For multiprocessors For multiprocessors Implementing Implementing Basic Test algorithm Basic Test algorithm The algorithm The algorithm

Work Done Studied Background mixture Model for foreground subtraction algorithm [2], [3], as a case study. Studied Background mixture Model for foreground subtraction algorithm [2], [3], as a case study. Analysis of the algorithm for: Analysis of the algorithm for: Parallelism exploitation Parallelism exploitation Length of code for implementation Length of code for implementation Memory requirements to store data. Memory requirements to store data. Feasibility Feasibility

Work Done Camera Video ADC` Virtex II Pro RGB Conversion Power PC Video DAC Monitor Top Down Approach

Work Done Studied Microblaze architecture. [4] Studied Microblaze architecture. [4] Studied Studied FSL Link [5] FSL Link [5] PLB, LMB, OPB Buses [6] PLB, LMB, OPB Buses [6] XPS Design Flow [7] XPS Design Flow [7] Literary survey on related works [8],[9], [10], [11] Literary survey on related works [8],[9], [10], [11] Configuration Video input and output for XUPV2P FPGA kit. Configuration Video input and output for XUPV2P FPGA kit. Bottom Up Approach

Work Ahead – Step 1 Camera Video ADC` Virtex II Pro RGB Conversion Power PC MEMORYMEMORY Video DAC MPMC Monitor Memory Read And Write 23 rd Feb – 7 th March

Work Ahead – Step 2 Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 MEMORYMEMORY Video DAC MPMC Monitor Some Simple Processing 8 th March – 15 th March

Work Ahead – Step 3 Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 MEMORYMEMORY Video DAC MPMC Monitor Simple processing 24 th March – 5 th April

Work Ahead – Step 4 Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 MEMORYMEMORY Video DAC MPMC Monitor Complex Processing 6 th April – 19 th April

Time Line Step 1 – 23 rd Feb – 7 th March Step 1 – 23 rd Feb – 7 th March Step 2 – 8 th March – 15 th March Step 2 – 8 th March – 15 th March Step 3 – 24 th March – 5 th April Step 3 – 24 th March – 5 th April Step 4 – 6 th April – 19 th April Step 4 – 6 th April – 19 th April

Related Works Design Development and performance evaluation of multiprocessor system on FPGA. Somen Barma, CSE IITD. [8]. Design Development and performance evaluation of multiprocessor system on FPGA. Somen Barma, CSE IITD. [8]. A Microblaze based Multiprocessor SoC[1] A Microblaze based Multiprocessor SoC[1] An FPGA based soft multiprocessor system for IPv4 packet forwarding. [9] An FPGA based soft multiprocessor system for IPv4 packet forwarding. [9] An automated framework for FPGA based soft Multiprocessor System. [10] An automated framework for FPGA based soft Multiprocessor System. [10] Multiprocessor interconnection based on DMA for FPGA.[11] Multiprocessor interconnection based on DMA for FPGA.[11]

REFERENCES [1] A Microblaze based Multiprocessor SOC – 2003 [2] Adaptive background mixture model for real-time tracking – 1999 [3] Understanding background mixture model for foreground segmentation – 2002 [4] Microblaze processor reference guide [5] Xilinx FSL datasheet [6] Xilinx Microblaze bus interface (ppt) [7] Virtex II Pro design flow – getting started [8] Design Development and performance evaluation of multiprocessor system on FPGA. Somen Barma, CSE IITD. Under Prof Kolin Paul [9] An FPGA based soft multiprocessor system for IPv4 packet forwarding. [10] An automated framework for FPGA based soft Multiprocessor System. [11] Multiprocessor interconnection based on DMA for FPGA. [12] XPS White paper – Designing multiprocessor System on Platform Stdio. Visit