1. Peter Tummeltshammer, Martin Delvai
HW-SW Codesign
Lab Course Setup
Peter Tummeltshammer,
Martin Delvai

2. Overview Face recognition Implementation details Further Steps
How does it work
Lab course approach
Implementation details
Further Steps
ECS Group Tummeltshammer & Delvai

3. Facial Recognition System
Computer-driven application
Identifies a person from a digital image
Identification vs. Verification
Identification: Comparing to a single image
Verification: Comparing to a database
Not to confuse with face detection
Locate a face inside an image
ECS Group Tummeltshammer & Delvai

4. Known FR-Algorithms 2D Face Recognition 3D-Face recognition Eigenfaces
Correlation filters
Hidden Markov Model
Dynamic Link Matching
3D-Face recognition
ECS Group Tummeltshammer & Delvai

5. Image Correlation 2D FFT / IFFT Point-wise multiplication
Peak to Sidelobe Ratio (PSR) calculation
ECS Group Tummeltshammer & Delvai

6. Input Images 32x32 pixels 8 bit greyscale No/only little background
Consistent lightning
High contrast
ECS Group Tummeltshammer & Delvai

7. … … 2D FFT decomposition Can be decomposed into 1D FFTs
1. 1D 32-point FFT of all 32 lines
2. 1D 32-point FFT of all 32 columns
Overall point FFTs … …
ECS Group Tummeltshammer & Delvai

8. 2D FFT result Result: Complex 32x32 Matrix
Problem for fixed-point precision: big numerical range
real
imag
ECS Group Tummeltshammer & Delvai

9. 2D FFT Decomposes image data into frequency domain
Representation of facial feature transitions
Constant component at (0,0)
ECS Group Tummeltshammer & Delvai

10. Pointwise Multiplication
Complex point-wise multiplication
(a+bi) * (c+di)=ac+adi+bci-bd
Four Multiplications for each pixel
Overall 4096 Multiplications
ECS Group Tummeltshammer & Delvai

11. 2D inverse FFT (IFFT) Similar to 2D FFT (conjugated complex)
64 32-point 1D IFFTs
Only real part necessary for PSR calculation
ECS Group Tummeltshammer & Delvai

12. Peak to Sidelobe Ratio PSR is a measure for correlation 1. Find Peak
2. draw 5x5 and 21x21 rectangles (sidelobe region)
3. calculate µ and σ inside sidelobe region 21 5 32 32
ECS Group Tummeltshammer & Delvai

13. Peak to Signal Ratio High PSR = high correlation
PSR > Θ -> Person identified
Θ is application- and filter specific
PSR=38
PSR=3
ECS Group Tummeltshammer & Delvai

14. Correlation in Face Recognition
Offline Generation of reference filter
One correlation filter for each person
Out of N training images
Filters are stored in frequency domain
Online
Offline
ECS Group Tummeltshammer & Delvai

15. Verification Example PSR 1 ≤ Θ PSR 2 ≤ Θ … PSR N > Θ
VerifyImage(image, filterdatabase) {
fftimage = FFT(image);
for (i=0; i<size(filterdatabase); i++) {
multimage = fftimage * filterdatabase[i];
ifftimage = ifft(multimage);
PSR = computePSR(ifftimage);
if (PSR > Θ) {
return i; }
return -1;
PSR 1 ≤ Θ
PSR 2 ≤ Θ …
PSR N > Θ
ECS Group Tummeltshammer & Delvai

16. MACE Filter Design
Minimum Average Correlation Energy Filter [Savvides et al.]
High illumination invariance
Generates sharp peaks in PSR analysis
Filter generation performed offline
For details: [Savvides et al.] or MATLAB
ECS Group Tummeltshammer & Delvai

17.   Design Path MATLAB Model Algorithmic approach
Floating point precision
Provided by us
C- Implementation
Fixed point precision
Provided by us
HWSW-CD Partitioning
Timing optimization
Todo by you
ECS Group Tummeltshammer & Delvai

18. MATLAB Model Available on HW-SW CD Homepage
Optional (only if interested)
Workspace (facerec.mat)
Images (10 example images)
Filter (single correlation Filter)
Functions
computePSR
generateFilter
ECS Group Tummeltshammer & Delvai

19. MATLAB Model Example command:
computepsr(real(ifft2(fft2(double(Images(:,:,:,i))) .* filterN)));
Computes PSR of filterN and Image i inside Vector “Images“
Ifft2 and fft2 are MATLAB functions that need double precision input
Computepsr returns PSR of given matrix
ECS Group Tummeltshammer & Delvai

20. C Implementation Available on HW-SW CD Homepage Mandatory download
Calculates PSR for given image and filter
Uses 16 bit fixed point representation
ECS Group Tummeltshammer & Delvai

21. C Implementation Uses fixed point representation Challenges:
Most arithmetic operations can cause an overflow
Requires scaling
Fixed scaling -> loss of precision
Pro: dynamic scaling won‘t affect the PSR calculation due to division by standard deviation
ECS Group Tummeltshammer & Delvai

22. C Filestructure Compile with make (gcc)
/facerec
frconsole.c
fix_fft.c
Makefile
filter_2_4_9.txt
image1.txt
image2.txt
image10.txt …
Compile with make (gcc)
Filter generated from images 2, 4 and 9 (same as MATLAB)
Syntax:
./frconsole image1.txt filter_2_4_9.txt
./frconsole image2.txt filter_2_4_9.txt …
ECS Group Tummeltshammer & Delvai

23. C Implementation FFT taken from web (http://www.jjj.de/)
FFT scales automatically
PW multiplication and PSR calculation scale dynamically
Images and filter are scaled to 16 bit integer [-32768, 32767]
ECS Group Tummeltshammer & Delvai

24. HW-SW CD Partitioning Implemented on ALTERA FPGA, using Softcore SPEAR
16 bit Instruction & Data Memory
No FPU, no Multiplication
Integration of custom HW-modules by memory mapping
Concept presentation on
ECS Group Tummeltshammer & Delvai

25. HW-SW CD Partitioning Timing optimization by
Analyzing code for repetative and time- consuming function calls
Implementing HW-modules for these functions
Implementing whole blocks (i.e. like 2D- FFT) in HW
ECS Group Tummeltshammer & Delvai

26. Further Steps 1.: Download and understand C-Code
2.(optional): Download MATLAB design
3.: Compile C-code for SPEAR (SPEAR concept presentation: )
4.: Analyze time consumption
5.: Implement HW modules and partition design
ECS Group Tummeltshammer & Delvai

27. Questions?
ECS Group Tummeltshammer & Delvai

28. Organisatorisches Gruppenbildung Email in TI System
Fragen zu C-Code: Mi :00 – 14:00
ECS Group Tummeltshammer & Delvai