1. Final Project presentation
Design space exploration and optimization of FPGA-based Neural Network Accelerator
Final Project presentation
Maedeh Hemmat
ECE734 -  VLSI Array Processors for Digital Signal Processing

2. Goal: For any CNN on any arbitrary FPGA platform, find a mapping that:
Outline
Goal: For any CNN on any arbitrary FPGA platform, find a mapping that:
Minimizes the data transfer between computational units and memory
Maximizes data reuses
Increases the throughput of the accelerator
Approach and Progress: A software-hardware co-design
Software: Developing an algorithm to explore the design space by considering the structure of networks and characteristics of CNN
Hardware: Implementing the optimal mapping in hardware (ModelSim)
Measure power consumption and throughput of the network under different scenarios
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited

3. Motivation
Input feature Map:
𝑁 𝑥 × 𝑁 𝑦 × 𝑁 𝑖𝑓
Convolutional layer:
( 𝐾 𝑥 × 𝐾 𝑦 × 𝑁 𝑖𝑓 × 𝑁 𝑜𝑓 )
output feature Map:
𝑁 𝑥′ × 𝑁 𝑦′ × 𝑁 𝑜𝑓
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited
The four-nested loop leads to a large design space with various choices to implement the CONV layer while exploiting parallelism

4. Proposed Algorithm What are the choices? Which one is the optimal one?
Unroll over input feature maps (Channels)
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited
Processing 𝑃 𝑖𝑓 channels at one clock cycle
𝑃 𝑖𝑓 pixels from different kernels/feature maps are used
An accumulator is required to accumulate the results from different channels
If 𝑃 𝑖𝑓 = 𝑁 𝑖𝑓 => one output /clock cycle

5. Proposed Algorithm What are the choices? Which one is the optimal one?
Unroll over output feature maps (Kernels)
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited
Processing 𝑃 𝑜𝑓 kernels at one clock cycle
Input pixels from same feature map are multiplied with weights from different kernels
Each input pixel is reused for 𝑃 𝑜𝑓 times
- Partial results over channels need to be stored

6. Pick the mapping with less latency and less on-chip buffer access
Proposed Algorithm
Pick the mapping with less latency and less on-chip buffer access
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited

7. Experimental Results
Trade off between different mapping under different number of channels
Size of weight matrix : 5 x 5 x Number_of_channels x 32
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited

8. Experimental Results
Trade off between different mapping under different number of channels
Size of weight matrix : 5 x 5 x Number_of_channels x 32
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited

9. Number of kernels per Tile Output pixels per cycle
Experimental Results
Target CNN: LeNet5
Two convolutional layer followed by one fully connected layer
Trained and tested on MNIST data
First Conv layer : 5x5x1x20 (Unrolling over channels)
Kernel Tiling Factor
Number of kernels per Tile
On-chip buffer access
Output pixels per cycle 1 20
525 10 4 5
1025 8 3
1225
1.5
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited
Second Conv layer: 5x5x20x50

10. Hardware Simulation
Implementing the second layer of LeNet5 in hardware
16-bit fixed point operation
Two-input multipliers and adder trees to implement MAC
Quantizing the generated output to 16-bit to control dynamic range growth
Using Modelsim to implement the design
Synthesizing the design at 45 nm technology
Measuring the power consumption of the layer under different scenarios ( by changing the available on-chip buffer size)
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited

11. Experimental Results
Power Consumption of second layer of LeNet5 under different scenarios
NOwadays, CNNs have gained significant attentoiin to perform more complicated tasks such as To be able to perform these complicated tasks, neural networks are becomming deeper and deeper, that is infact the increase in number of layers.
Deeper neural network makes the power-efficient hardware implementation more challenging, especialy when it comes to embedded devices, cell phones, and wearable debices where the energy budget is limited
The on-chip buffer size varies from 40kb to 8 kb

12. Conclusion
Developing an algorithm which finds the optimal mapping of CNN on FPGA platforms
Minimizing the number of on-chip buffer accesses to minimize power consumption
Implementing and synthesizing one layer of LeNet5

13. Q & A