1. General
Engineering
Research
Institute
A Real-Time Multi-Sensor 3D Shape Surface Measurement System Using Fringe Analysis
By Mohammad Al Sa’d

2. Introduction General background Functional requirements of the system
Stages of the 3D surface reconstruction process
Specifications of the system
Hardware design of the system
Software design of the system
Results

3. Background (1/2) Fringe Pattern Profilometry
Projector
Object
Optical non-contact 3D surface method
Fringes generation
Laser Interference
structured light projection
Measurement precision: from 1μm
Depends on the optical resolution of the fringes
Fringes width and their optical quality (depth of field, camera resolution and display resolution)
Light Wavelength
Applications
Inspection of components during the production process (turbine blades and circuit boards)
Reverse engineering (CAD data from existing objects)
Documenting objects of cultural heritage
Medical applications: live measuring of human body shape
Camera
Projected Pattern
Image plane
patterns are generated by computer and displayed by a projector

4. Background (2/2) Metrology Guided Radiotherapy
Radiation therapy is used since about more than 100 years for the treatment of cancer.
The goal is to destruct the cancer cells with minimal radiation damage to the surrounding healthy cells.
Pre-treatment stages:
3D planning models are created (CT, MR or others) to accurately guide treatment.
Radiation treatment sessions are planned and radiation doses are calculated (dosimetry).
Treatment stages:
Radiation beam is shaped to precisely hit the target (site of tumour).
Radiation is delivered from multiple angles, using the controlled gantry and patient table.
Treatment is repeated over multiple sessions.
Any small movement (like breathing) or patient’s body changes during successive sessions affect the goal of the treatment.
Rotating Gantry
The goal is to destruct the cancer cells with minimal radiation damage to the surrounding healthy cells

5. Functional Requirements
400mm Z Y X
Field of View
At least 400mm × 400mm × 400mm
Spatial Resolution
At least 100 x,y points
Measurement Error (Accuracy)
Not to exceed ±1 mm (according to the tolerance of the dosimteric models used in radiotherapy planning).
Dynamic Real-Time Measurement
At least FIVE measurements per second (to detect small movements, like breathing)
Patients are asked to be static as possible, however movement in their bodies come from breathing

6. 3D Surface Reconstruction Stages (1/5)
Fringe Profilometry Analyses
Phase Unwrapping
Calibration

7. 3D Surface Reconstruction Stages (2/5)
Fringe Profilometry Analyses
Spatial Fringe Analysis Techniques (modulation phase is generated from a single input image):
Fourier Profilometry:
Windowed Fourier Profilometry: Processing window passes through the image to find the phase at the centre pixel using forward and inverse FFT.
Wavelet Profilometry: Generating phase from the wavelets of the image line- by-line
Temporal Fringe Analysis Techniques
(modulation phase is generated from multiple
input images – at least three images):
Phase-Stepping Profilometry: Using least square
method to extract the phase.
Forward FFT
Filter out the fringes
Inverse FFT
Calculated Phase 1 3 2

8. 3D Surface Reconstruction Stages (3/5)
Phase Unwrapping
To remove the phase ambiguity (2 modules).
Types:
Path-Dependent Unwrappers
Schafer Unwrapping Algorithm
Path-Independent Unwrappers
Goldstein's Branch Cut Algorithm
Quality-Guided Path Following Algorithm
Flynn's Minimum Discontinuity Algorithm
Preconditioned Conjugate Gradient (PCG) Algorithm
Lp-Norm Algorithm
Reliability Ordering Algorithm
Synthesis Algorithm
Differ in speed and robustness.

9. 3D Surface Reconstruction Stages (4/5)
Absolute Height Calibration
Unwrapped phase map is converted to real world heights
Height calibration process:
Triangulation spot (embedded inside the fringe
pattern) is detected.
Unwrapped phase value at the spot (x,y)
location is subtracted from the unwrapped
phase map (to generate a relative phase map).
Unwrapped phase map is linked to the real world heights via interpolation (using the height calibration volume).
To compensate the geometric distortions by optics and perspective.
X,Y world coordinates are generated for a number of
height steps to generate the traversal calibration volumes
X,Y world coordinates are retrieved depending on the
correspondent height value of each pixel and using
interpolation.
Traversal (XY) Calibration

10. Specifications of the System (1/3)
Deliverables (so far!)
Speed: 8Hz (using Fourier Profilometry and Goldstein's unwrapper)
5Hz (using Fourier Profilometry and Reliability ordering unwrapper)
Field-of-view:
(X,Y,Z) = 400mm × 500mm × 400mm
Spatial resolution:
262,144 x,y points
Multiple sensors:
Coverage area around 270°
Measurement error:
Accuracy around ±0.5 mm
Pre-processing Techniques:
Noise Reduction and gamma Correction
Catalogue of measurement techniques:
Ability to select different algorithms
User interaction and multi-user modes:
GUI to interact with the user
Normal and advanced modes (for both metrology experts and normal users)
Various operating modes:
Online: for real-time measurements
Offline: for pre-saved images and videos
3D visualisation and 2D plotting.
Various image saving choices.
Treatment coach
Sensor2
Sensor3
Sensor1

11. Specifications of the System (2/3)
Program snapshots

12. Specifications of the System (3/3)
Program snapshots

13. Hardware Design of the System (1/3)
Hardware Configuration
Main Control & Processing Unit
Synchronisation Unit
Sensor Processing Unit
Sensor2
Sensor3
Sensor1
Sensor Processing Unit
Sensor Processing Unit

14. Hardware Design of the System (2/3)
Sensor Components: Projector
Canon XEED SX60
Conventional LCD projector
LCoS projector

15. Hardware Design of the System (3/3)
Sensor Components: Camera
Gige Broadcasting
Phase-Stepping
Triangulation
Fourier
Controller
Gige Camera
Prosilica GE1380
GigE technology:
Progressive scan CCD
× 1024
x 512
Direct image registration to the system memory via a compatible Gigabit port
Upto 100 meter cable length

16. Software Design of the System
Software Configuration: Processing Core
Multithreaded processing framework:
Input Thread: Project and grab frames
Processing thread: Apply measurement, unwrapping and calibration techniques
Output thread: Stream/save/display results
Grabbed Images from the Camera
Input Thread
Input Buffer
Processing Thread
Output Buffer
Output Thread
Display and/or store according to the user preferences

17. Results (1/3)
Static Object Measurement – One Sensor

18. Results (2/3) Static Object Measurement – Multi-Sensor Sensor1 Sensor2
Treatment coach
Sensor2
Sensor3
Sensor1

19. Results (3/3)
Moving Object Measurement – One Sensor

20. Thank You!